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Foreword 


This  project  was  sponsored  by  the  Office  of  Naval  Research  through  the 
Independent  Laboratory  Independent  Research  (ILIR)  program.  The  ILIR 
program  makes  grants  to  Navy  research  laboratories  to  allow  researchers  to 
pursue  interesting  theoretical  research  on  topics  that  they  might  not  otherwise 
receive  funding  for.  The  present  study  was  conducted  between  October  2001  and 
September  2002;  the  study  sought  to  compare  traditional  face-to-face  with  two 
computer-based  focus  group  methods  to  evaluate  the  differing  levels  of  social 
presence  and  the  overall  success  of  the  approach.  The  primary  reasons  for  this 
study  were  two-fold,  to  test  for  differences  between  the  quantity  and  quality  of 
focus  group  responses  in  both  face-to-face  (FTF)  and  computer-mediated  (CM) 
conditions,  and  to  evaluate  the  impact  of  social  presence  on  the  focus  group 
process  and  outcomes.  The  study  successfully  completed  27  focus  groups,  using  a 
sample  of  156  junior  enlisted  and  petty  officers  currently  in  training  at  the  NTC 
Great  Lakes,  to  evaluate  differences  in  the  success  of  FTF  and  CM  focus  groups. 
Participants  in  randomly  assigned  groups  of  5-6  participated  in  one  of  the  three 
following  conditions:  traditional  face-to-face  (standard  focus  group  utilizing 
verbal  and  non-verbal  communication);  computer-mediated  (group  members  in 
same  location  but  all  communication  via  computer);  and  Internet-simulated 
(group  members  in  same  location  but  separated  by  barriers  and  noise  canceling 
headphones,  with  all  communication  via  computer). 

While  differences  were  found  in  the  number  of  words  used  to  communicate, 
the  three  conditions  appeared  to  achieve  similar  results  in  level  of  participation, 
participant  satisfaction,  and  quantity/ quality  of  information  obtained.  In  contrast 
to  what  was  expected,  social  presence  did  not  appear  to  clearly  distinguish 
between  each  of  the  different  modes  of  participation.  Overall  the  results  indicate 
that  conducting  on-line  focus  groups  can  be  done  in  a  successful  manner  and 
achieve  similar  results  as  typically  found  in  traditional  face-to-face  methods. 
While  there  are  some  differences  in  the  amount  of  conflict  and  words  used  to 
communicate,  similar  amounts  of  unique  information  are  obtained  for  both  FTF 
and  CM  focus  groups. 


DAVID  L.  ALDERTON,  Ph.D. 

Director 
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This  project  was  sponsored  by  the  Office  of  Naval  Research  through  the  Independent 
Laboratory  Independent  Research  (ILIR)  program.  The  ILIR  program  makes  grants  to 
Navy  research  laboratories  to  allow  researchers  to  pursue  interesting  theoretical 
research  on  topics  for  which  they  might  not  otherwise  receive  funding.  The  grants  are 
locally  administered  by  a  project  officer  who  reviews  and  approves  a  number  of  grants 
each  year  to  researchers  within  the  lab.  To  be  considered  for  the  grant,  the  study  must 
focus  on  evaluation  of  a  theoretical  issue  that  has  some  potential  practical  implications. 
At  the  conclusion  of  ILIR  studies,  the  researcher  is  required  to  submit  at  minimum  a 
brief  description  of  the  results  of  the  study.  This  Annotated  Briefing  provides  details  on 
the  background,  results,  and  implications  of  the  present  study  as  required  by  the 
program. 

The  present  study  was  conducted  between  October  2001  and  September  2002.  The 
grant  was  managed  in-house  by  Murrey  Olmsted,  while  the  research  was  conducted  by 
both  Murrey  Olmsted  and  Christina  Underhill.  This  project  served  a  dual  purpose,  in 
that  it  provided  valuable  research  on  the  process  of  conducting  focus  groups  on-line  and 
also  served  as  a  data  collection  vehicle  for  a  Master’s  thesis  for  Christina  Underhill.  Ms. 
Underhill  successfully  defended  her  Master’s  thesis  in  psychology  at  the  University  of 
Memphis  in  December  of  2002. 
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•  Navy  Marine  Corp  I  ntranet  (NMCI)  and  IT-21  promise  new  and 
widely  available  Internet  access 

Improvements  in  networks  and  computer  equipment  will  allow  for  near 
universal  Internet  coverage 

Increased  bandwidth  of  Internet  communications  promises  to  deliver  real¬ 
time  text,  sound,  and  video  between  parties  on  NMCI 

•  NPRDC/NPRST  research  roadmap  called  Sailor  21 
(http://www.NPRST.navv.mil/Sailor.htm)  envisioned  the 
Internet  as  an  important  new  medium  for  collecting  research 
data 

Sailor  21  recommends  a  major  shift  in  focus  in  research  methods  to  exploit 
new  technologies  such  as  the  Internet,  wireless  communication,  and  others 
which  may  provide  new  approaches  to  collecting  valid  and  reliable  data 
from  Sailors 

New  media  would  not  only  be  used  to  collect  data,  but  may  also  provide  a 
platform  for  publishing  the  results  of  these  studies 
Both  in  data  collection  and  dissemination,  a  great  deal  of  research  needs  to 
be  done  on  the  feasibility,  validity  and  reliability  of  these  methods  to  ensure 
that  they  are  viable  in  the  Navy 
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Navy  IT  Initiatives.  During  the  mid-1990s,  the  Navy  began  exploring  the  need  to 
develop  a  comprehensive  and  integrated  network  infrastructure  that  offered  universal 
connectivity  and  standard  operations  for  all  active  duty  and  civilian  personnel  working 
for  the  Department  of  Navy.  The  results  of  this  work  were  two  independent  but 
complimentary  visions  for  building  a  communications  infrastructure  for  the  Navy.  The 
first  of  these  plans  resulted  in  the  development,  and  now  partial  implementation,  of  the 
Navy/Marine  Corps  Intranet  (NMCI).  NMCI  is  designed  to  provide  consistent  and 
universal  Internet  and  Intranet  access  to  all  personnel  at  shore-based  facilities.  The 
access  provided  by  the  network  is  designed  to  provide  a  consistent  “look  and  feel”  for 
users  as  well  as  a  suite  of  applications  and  services  necessary  to  facilitate  normal 
operations. 

The  other  network  initiative,  the  Information  Technology  for  the  21st  Century 
(IT-21),  is  a  program  to  upgrade  the  communications  and  information  technology  of 
deployable  or  sea-going  units.  The  focus  of  this  initiative  is  to  upgrade  ship-based 
computer  and  communications  networks  to  the  latest  industry  standards  aboard  ships 
and  other  deployable  military  platforms.  IT-21  was  originally  a  voluntary  initiative  to 
encourage  commands  to  upgrade  equipment  to  the  latest  industry  standards  and  to 
coordinate  this  upgrade  in  technology  across  the  Navy.  The  initiative  is  now  considered 
by  some  to  be  the  equivalent  of  the  NMCI  program;  however,  the  focus  is  on  deployable 
units  vs.  shore-based  units. 

Sailor  21.  In  December  1998,  the  Navy  Personnel  Research  and  Development  Center 
(NPRDC)  published  a  visionary  road  map  of  how  the  Navy  could  work  to  achieve  its 
manpower  and  personnel  research,  staffing,  and  management  needs  through  research 
and  development  (NPRDC,  1998).  The  document,  titled  Sailor  21,  cast  a  vision  for  the 
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future  of  manpower  and  personnel  research  in  the  Navy.  One  of  the  fundamental 
changes  to  research  introduced  by  Sailor  21  was  a  focus  on  using  technology— 
specifically  the  Internet— as  a  tool  for  data  collection,  reporting  results,  and  conducting 
follow-up  research.  Since  its  introduction,  Sailor  21  has  been  widely  briefed  to  senior 
leadership  throughout  the  Navy.  This  has  resulted  in  fairly  wide  adoption  of  the  ideas 
expressed  in  the  research  roadmap  by  senior  leadership  as  being  the  vision  for  research 
and  development  in  areas  related  to  Navy  manpower  and  personnel. 

Integration  of  IT  Initiatives  and  Sailor  21.  Together,  the  current  Navy  IT  initiatives 
(NMCI  and  IT-21)  and  Sailor  21  point  toward  doing  more  research  on  and  through  the 
Internet.  They  laid  the  groundwork  for  a  major  revolution  in  how  research  is  conducted 
for  the  Navy.  In  the  realm  of  social  and  public  opinion  research,  the  Internet  offers  a 
new  mode  of  collecting  data  from  web-based  surveys,  opinion  polls,  focus  groups,  and 
other  methods  of  research.  They  also  point  to  the  aggressive  adoption  of  technology- 
based  research  methods  as  a  means  to  improve  the  access,  speed,  and  accuracy  of 
research. 
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Computer-mediated  vs.  Face-to- 
face  Communication 


•  Equality  of  Participation 

Reviews  by  McLeod  (1992)  and  Bordia  (1997)  indicate  that  there  is 
generally  greater  equality  in  the  amount  of  participation  when 
conducting  computer-mediated  forms  of  communication 

•  Quantity  of  Information 

-  Studies  by  Parent  et  al.  (2000),  Olaniran  (1994),  and  Bordia  (1997) 
suggest  that  computer-mediated  communication  typically  generates 
a  greater  quantity  of  information  (i.e.,  unique  comments) 

•  Quality  of  I  nformation 

Weisband  et  al.  (1995)  and  Bordia  (1997)  found  that  the  quality  of 
information  obtained  in  computer-mediated  communication  tends 
to  be  mixed 

•  Participant  Satisfaction 

According  to  Parent  et  al.  (2000)  and  Olaniran  (1996),  participant 
satisfaction  was  greater  with  face-to-face  rather  than  computer- 
mediated  communication 
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As  computers  and  other  new  communication  technologies  have  become  a  ubiquitous 
part  of  modern  life,  the  question  of  their  equivalence  in  producing  usable  research  data 
has  come  to  the  forefront.  A  number  of  studies  have  been  published  comparing 
computer-mediated  (CM)  and  face-to-face  (FTF)  communication.  The  major  areas  these 
studies  have  focused  on  have  been  equality  of  participation,  quantity  and  quality  of 
information,  and  participant  satisfaction. 

Equality  of  Participation.  When  in  a  computer-mediated  communication 
environment,  status  differences  are  difficult  to  determine  (Weisband,  Schneider,  & 
Connolly,  1995).  Unlike  FTF  communication,  where  social  cues  are  often  a  major  part  of 
the  communication  taking  place,  in  CM  less  attention  is  given  to  social  factors  and  more 
attention  paid  to  the  actual  ideas  being  communicated.  According  to  reviews  of  the 
literature  by  McLeod  (1992)  and  Bordia  (1997)  participants  are  less  likely  to  dominate 
the  group  discussion  in  CM  communication  resulting  in  greater  equality  of  participation 
across  groups.  Straus  (1997)  found  a  greater  equality  of  participation  in  CM 
communication,  but  this  equality  did  not  have  an  impact  on  productivity  (quantity  or 
quality  of  ideas  generated)  or  satisfaction  with  the  discussion  process. 

Quantity  of  Information.  A  number  of  studies  indicate  that  CM  communication 
typically  generates  a  greater  quantity  of  information  (e.g.,  unique  comments)  than 
traditional  face-to-face  methods.  A  study  by  Parent,  Gallupe,  Salisbury,  and  Handelman 
(2000)  found  that  participants  using  electronic  group  support  systems  (a  type  of 
computer-mediated  communication)  produced  55  percent  more  unique  ideas  than  those 
using  more  traditional  methods  of  communication  (face-to-face,  telephone  support, 
etc.).  CM  methods  of  communication  allow  for  individuals  to  simultaneously  participate 
by  interjecting  their  comments  at  any  time  in  the  discussion  without  waiting  for  others 
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to  finish.  Other  studies  involving  idea  generation  tasks  (e.g.,  brainstorming)  found  that 
CM  produced  more  unique  ideas  than  face-to-face  discussion  (Bordia,  1997;  Olaniran, 

1994) .  In  addition,  literature  reviews  by  Bordia  (1997)  and  McLeod  (1992)  found  that 
there  is  typically  less  conformity  in  CM,  allowing  more  individual  contributions  and  a 
greater  quantity  of  unique  comments  to  be  generated. 

Quality  of  Information.  Published  reviews  of  the  research  literature  indicate  that  the 
quality  of  information  (e.g.,  ratio  of  on-topic  to  off-topic  comments)  obtained  in 
computer-mediated  communication  tends  to  be  mixed  (Bordia,  1997;  Weisband  et  al., 

1995) .  In  some  cases,  such  as  McLeod  (1992),  research  has  found  that  the  quality  of 
information  has  been  higher  for  CM  than  FTF  communication.  Other  authors,  such  as 
Bordia  (1997),  have  concluded  that  the  results  of  these  studies  are  still  rather  mixed  in 
terms  of  CM  and  FTF  equality  when  it  comes  to  information  quality.  A  few  more  recent 
studies  of  CM  used  in  the  process  of  brainstorming  were  unable  to  distinguish  between 
the  quality  of  information  found  in  CM  and  FTF  methods  of  data  collection  (Barki  & 
Pinsonneault,  2001;  Ziegler,  Diehl,  &Zijlstra,  2000). 

Participant  Satisfaction.  Participation  in  CM  communication  requires  significantly 
greater  complexity  than  FTF  communication  (Olaniran,  1996).  Participants  must  use 
things  such  as  a  keyboard  and  mouse,  read  the  computer  screen,  type  a  response,  or 
engage  in  other  activities  rather  than  simply  listening  and  talking  with  others. 
Consistent  with  what  might  be  expected  with  the  increased  workload  of  responding  in 
CM  communication,  published  studies  indicate  that  participant  satisfaction  tends  to  be 
greater  with  FTF  rather  than  CM  communication  (Gunawardena  &  Zittle,  1997;  Parent 
et  al.,  2000;  Tu,  2002;  2000). 
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•  Definition 

-  The  degree  to  which  a  particular  communication  medium  is 
successful  in  transmitting  both  content  information  and  social  cues 
that  typically  occur  in  face-to-face  communication  (Short  et  al., 
1976) 

•  Lack  of  On-line  Research 

Majority  of  past  social  presence  research  has  focused  on  face-to- 
face,  telephone,  video,  or  written  communication— almost  nothing 
has  been  done  with  on-line  communication 

•  Participant  Satisfaction 

-  Gunawardena  and  Zittle  (1997),  and  Tu  (2002;  &  2000)  found  that 
social  presence  was  a  strong  predictor  of  participant  satisfaction 
with  computer-mediated  communication 
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One  of  the  theories  that  may  be  helpful  in  understanding  differences  between  CM 
and  FTF  communication  is  social  presence.  Developed  in  the  mid-1970s  by  Short, 
Williams,  and  Christie  (1976),  the  theory  of  social  presence  is  a  framework  to  describe 
changes  in  the  perception  of  social  factors  with  new  types  of  technology-based 
communication.  Social  presence  is  defined  as  the  degree  to  which  a  particular 
communication  medium  is  successful  in  transmitting  both  content  information  and 
social  cues  that  typically  occur  in  face-to-face  communication  (Short  et  al.,  1976).  Social 
presence  can  be  conveyed  through  written  and  verbal  conversations  as  well  as  nonverbal 
social  and  symbolic  cues.  Each  different  type  of  media  such  as  telephone,  e-mail, 
videoconference,  and  personal  contact  differ  somewhat  in  their  ability  to  transmit  these 
cues  in  the  process  of  communication.  A  principle  underlying  social  presence  theory  is 
that  a  good  match  between  the  communication  behaviors/activities  of  a  particular  type 
of  media  and  its  ability  to  convey  the  presence  of  other  participants  will  produce  a  more 
effective  and  satisfying  experience.  For  example,  a  match  between  a  medium  that  has  a 
relatively  high  degree  of  social  presence  (e.g.,  using  the  telephone)  and  the  activity  of 
getting  to  know  someone  will  produce  more  satisfied  participants  (Rice,  1993). 

Social  presence  theory  offers  a  useful  perspective  and  basis  on  which  to  judge  new 
media  (e.g.,  types  of  technology-based  communication);  however,  very  little  research 
has  been  done  to  advance  theoretical  work  in  this  area  over  the  past  decade.  While  some 
early  research  considered  the  impact  of  social  presence  on  telephone  and  video 
communication  (Rice,  1993;  Short  et  al.,  1976)  very  little  has  been  done  regarding  its 
impact  on  computer-based  or  on-line  communication.  In  the  few  studies  that  have  been 
conducted,  most  have  found  that  social  presence  was  a  strong  predictor  of  satisfaction 
with  communication  in  a  CM  environment  (Gunawardena  &  Zittle,  1997;  Tu,  2002; 
2000).  The  higher  degree  of  social  presence  felt  by  the  participants,  the  greater  their 
satisfaction  with  CM  communication. 
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Focus  Groups 
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•  A  popular  qualitative  method  of  data  collection  in  the  Navy 

Focus  groups  are  frequently  used  in  the  Navy  as  a  means  of  both 
discussing  issues  and  generating  possible  solutions  to  problems 

-  Used  by  researchers,  policy-makers,  and  review  groups  (e.g.,  Navy 
Inspector  General)  to  gain  information  from  Sailors  about  topics  of 
interest 

•  A  common  method  of  conducting  qualitative  research 

-  Typically  involve  6-10  participants  and  a  trained  moderator(s) 

Draw  on  both  past  experiences  and  group  interaction  to  develop  a 
content  rich  source  of  data 

Often  used  in  both  preliminary  or  follow-up  stages  of  research  to 
generate  ideas  about  a  topic  or  clarify  issues 
Purpose  is  usually  to  explore  an  issue  in  depth  to  better  understand 
the  issues  (e.g.,  test  concepts,  develop  surveys,  brainstorm,  assess 
usability,  etc.) 
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Focus  groups  are  planned,  directed  discussions  involving  a  small  number  of  people 
that  concentrate  on  the  feelings,  beliefs,  and  perceptions  of  the  participants  on  a  pre¬ 
specified  topic  of  discussion  (Edmunds,  1999).  The  purpose  of  focus  groups  is  to  gather 
qualitative  information.  The  method  is  especially  helpful  in  providing  insight  into  how 
people  feel  or  think  about  an  issue,  product,  or  service.  A  focus  group  relies  on  both  the 
interactive  social  context  of  the  discussion  and  the  individual  histories  of  each  of  the 
group  members  to  produce  a  rich  and  interesting  discussion  on  a  particular  topic.  Group 
members  are  encouraged  to  talk  to  each  other,  react  to  the  comments  of  others,  and  talk 
about  their  past  experiences  that  have  a  bearing  on  the  discussion,  and  participate  by 
providing  other  comments  or  feedback  to  the  discussion.  Focus  groups  are  led  by  a 
moderator,  whose  job  it  is  to  maintain  the  focus  of  the  group,  ask  the  main  questions, 
follow  up  on  responses,  keep  notes,  and  maintain  a  time  schedule.  The  moderator  often 
records  group  discussions  so  that  they  can  be  transcribed  at  a  later  date.  An  effectively 
run  focus  group  can  provide  a  rich  source  of  data  on  a  specific  topic.  While  qualitative 
data  collected  in  focus  groups  may  not  be  generalized  to  larger  populations,  they  can 
still  shed  light  on  the  topic  of  focus  and  help  researchers  narrow  their  search  for  answers 
to  problems  (Edmunds,  1999;  Merton,  Fiske  &  Kendall,  1990;  Morgan  &  Krueger,  1998). 

Over  the  past  20  years,  focus  groups  have  become  a  popular  means  of  collecting 
qualitative  data  in  the  Navy.  They  are  currently  used  to  generate  data  on  a  wide  variety 
of  subjects  to  inform  decision-makers,  researchers  and  oversight  functions  (e.g.,  Navy 
Inspector  General)  in  the  Navy. 
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Benefits  of  On-line  Focus  Groups 
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•  Lower  overall  cost 

No  travel,  lodging,  or  other  labor  costs  associated  with  on-site  focus  groups 

•  Reduced  time  involvement 

Reduces  the  overall  amount  of  time  necessary  to  set  up  and  conduct  a 
focus  group 

•  Increased  accessibility 

Provides  high  accessibility  to  Sailors  in  remote  locations  who  may  be 
difficult  to  reach  using  on-site  data  collection 

•  Anonymity  of  participants 

Participant  identities  can  be  withheld  or  presented  in  a  manner  that 
protects  their  privacy 

•  Automatic  data  capture 

-  Software  enables  capture  of  all  discussion  content,  making  transcription  of 
the  results  unnecessary 

Enables  the  capture  of  information  on  the  response  patterns  of  focus  group 
participants  (i.e.,  key  strokes,  use  of  the  delete  key,  response  latency,  etc.) 
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In  today’s  work  world,  use  of  computers  for  both  work  and  communication  is 
widespread.  The  reality  of  frequent  computer-based  communication  is  true  for  both  the 
business  world  and  today’s  military.  In  the  Navy,  with  the  recent  push  to  move  more 
and  more  research  resources  to  the  Internet,  the  use  of  on-line  or  computer-based  focus 
groups  seems  to  be  a  logical  extension  of  both  the  research  methodology  and  the 
popularity  of  focus  groups.  Computer-based  focus  groups  are  now  also  feasible  for  the 
first  time  as  the  Sailor  population  has  increased  access  to  the  Internet  and  is  generally 
well  trained  in  the  use  of  computers  for  communication  and  other  business  functions. 
This  new  mode  of  research  offers  a  number  of  advantages  over  the  traditional  approach 
to  conducting  focus  groups  that  often  involved  extensive  travel,  time,  cost,  and  set-up 
for  each  location  (Edmunds,  1999;  Merton,  Fiske  &  Kendall,  1990). 

The  use  of  on-line  focus  groups  offers  a  number  of  significant  advantages  for  Navy 
researchers.  First,  the  overall  costs  associated  with  conducting  focus  groups  should  be 
much  less.  The  primary  reason  for  this  is  that  researchers  do  not  have  to  incur  the  costs 
of  travel,  lodging,  or  other  labor  associated  with  conducting  data  collection  on-site. 
Second,  on-line  focus  groups  require  less  time  to  conduct.  Moderators  do  not  have  to 
worry  about  scheduling  or  setting  up  a  room  or  travel,  and  may  use  the  same  protocol 
numerous  times  without  having  to  modify  it  for  different  settings.  Third,  on-line  focus 
groups  can  increase  the  accessibility  of  researchers  trying  to  get  to  study  participants.  As 
the  number  of  Sailors  with  access  to  the  Internet  increases,  a  wider  variety  of  Sailors 
from  around  the  Navy  can  participate  in  focus  groups,  surveys,  or  other  research  by 
simply  connecting  remotely  to  research  websites.  Fourth,  on-line  focus  groups  can  offer 
better  privacy  protections  than  would  be  typically  afforded  by  traditional  face-to-face 
focus  groups.  This  is  a  significant  advantage  in  that  study  participants  could  be  assured 
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that  there  would  not  be  any  negative  consequences  from  their  participation  in  the  group 
discussion.  Since  only  the  moderator  of  the  group  would  know  the  identity  of  the 
respondent,  he/she  would  be  free  to  express  their  opinions  and/or  dissent  without  fear 
of  reprisal.  Fifth,  on-line  focus  groups  offer  a  significant  advantage  to  researchers  by 
enabling  automatic  data  capture  of  the  group  discussion.  This  alone  makes  an  on-line 
focus  group  attractive  to  researchers  because  it  reduces  the  cost  and  time  to  conduct 
research  while  increasing  the  accuracy  of  data  collection.  Since  the  researcher  does  not 
need  to  transcribe  the  data,  he  can  immediately  move  into  the  analysis  phase  of  the 
study  and  start  to  evaluate  the  data  for  common  themes.  In  addition,  automatic  data 
capture  may  allow  researchers  to  conduct  research  on  the  response  patterns  of 
participants  using  measures  of  their  interaction  with  the  computer  during  the 
communication  process  (e.g.,  number  of  key  strokes,  use  of  the  delete  key,  response 
latency,  typing  speed,  etc.). 
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Disadvantages  of  On-line  Focus 
Groups 

> 

•  Absence  of  normal  visual  and  auditory  communication  cues 

Respondents  cannot  see  or  hear  each  other  because  they  are  in  different 
locations  and  only  have  access  to  the  text  transcript  of  the  discussion 

•  Limited  group  dynamics 

-  Since  participants  do  not  have  any  access  to  visual,  auditory,  or  social  cues 
on-line,  they  may  be  less  likely  to  provide  the  dynamic  discussion  that  is 
typical  of  focus  groups 

•  Additional  respondent  requirements 

Participants  are  required  to  be  able  to  read  and  type  their  responses  in  on¬ 
line  group  discussions.  This  involves  more  work  than  a  verbal  discussion 
and  may  cause  problems  as  people  differ  in  speed  of  reading  and  writing 
using  computers. 

•  Unequal  access  to  the  Internet 

May  obtain  a  biased  sample  because  not  everyone  has  access  to  the 
Internet 

•  Potentially  limited  discussions  or  topics 

Not  all  topics  may  be  suitable  for  on-line  focus  group  discussions  because 
of  participant  concerns  about  confidentiality  of  information  discussed 
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While  on-line  focus  groups  offer  a  number  of  advantages  over  traditional  face-to-face 
(FTF)  focus  groups,  they  also  present  a  number  of  disadvantages  (Edmunds,  1999).  The 
majority  of  these  disadvantages  are  due  to  differences  or  limitations  introduced  by  the 
use  of  computers  into  the  communication  process  in  on-line  focus  groups.  First,  because 
on-line  focus  groups  do  not  occur  in  the  same  location  and  typically  use  text-based 
communication,  participants  lose  out  on  all  of  the  visual  and  social  cues  that  are  present 
in  FTF  communication  to  convey  emphasis,  irony,  or  other  subtleties.  Instead, 
participants  must  rely  on  what  social  information  they  can  glean  from  the  words  being 
used  in  communication.  Second,  as  a  result  of  the  lack  of  visual  or  social  cues  being 
present  in  on-line  focus  groups  they  may  be  less  likely  to  produce  the  dynamic 
discussions  that  are  typical  of  FTF  focus  groups.  Third,  on-line  focus  groups  require 
participants  to  read  and  type  in  order  to  take  part  in  the  group  discussion.  This  adds  an 
additional  burden  to  the  respondent  to  both  read  the  computer  screen  quickly  and  then 
manually  type  the  response  using  a  keyboard.  For  most  people,  this  is  significantly  more 
work  than  simply  responding  verbally  and  as  a  result  most  tend  to  produce  shorter  more 
succinct  responses  to  questions  (Edmunds,  1999).  Fourth,  on-line  focus  groups  are  by 
definition  limited  to  those  who  have  access  to  the  Internet  or  some  other  computer- 
based  communication  network.  This  situation  presents  a  paradox  where  on-line  focus 
groups  can  both  improve  access  to  remotely  located  study  participants,  but  only  be 
restricted  to  those  with  Internet  access.  The  main  concern  with  this  situation  is  that 
there  may  be  systematic  differences  between  those  who  have  access  to  the  Internet  and 
those  who  do  not.  With  unequal  access,  the  probability  of  selection  for  inclusion  in  focus 
groups  and  possible  differences  in  attitudes  may  result  in  bias  that  would  potentially 
change  the  outcome  of  research.  Fifth,  as  a  result  of  the  limitations  imposed  by  using 
text-based  communication  in  on-line  focus  groups,  the  topics  of  discussion  may  be 
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limited.  For  instance,  discussions  of  highly  technical  topics  or  reactions  to  products/ 
services  may  simply  not  work  in  the  on-line  environment  due  to  the  lack  of  visual,  social 
and  other  communication  cues  or  aides. 
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State  of  the  Field 
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•  High  interest  in  on-line  focus  groups 

-  There  are  currently  over  50  commercial-off-the-shelf  software  titles 
available  that  offer  some  form  of  on-line  group  discussion  capability 
(e.g.,  chat-rooms,  instant  messaging,  etc.) 

-  There  are  a  number  of  companies,  such  as 

www.EFocusGroups.com  and  www.VRRoom.com.  that  currently 
offer  an  on-line  focus  group  service 

•  Lack  of  quality  published  research 

Majority  of  publications  focus  on  brainstorming  tasks  or 
asynchronous  discussions  (i.e.,  bulletin  boards),  and  not  real 
interactive  focus  groups 

Sample  sizes  have  been  very  low,  typically  2-4  groups,  leading  to 
very  low  statistical  power  (Cohen,  1992) 

-  Current  studies  talk  about  findings,  but  do  not  offer  any  supporting 
statistical  evidence  for  their  conclusions 
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At  the  present,  there  is  a  high  degree  of  interest  in  conducting  on-line  or  computer- 
mediated  (CM)  focus  group  research.  This  appears  to  be  due  in  part  to  the  advantages 
that  these  groups  offer  over  traditional  face-to-face  (FTF)  discussions.  As  a  result,  there 
are  a  large  number  of  software  titles  that  offer  some  form  of  on-line  group  discussion 
capability  such  as  chat  rooms,  instant  messenger,  shared  virtual  workspace,  and  other 
means  of  communicating.  A  recent  search  of  software  listing  services  such  as 
www.versiontracker.com  and  www.download.com  revealed  that  there  are  currently  over 
50  software  titles  offering  some  form  of  on-line  group  discussion  capability.  This 
indicates  that  there  is  a  great  deal  of  interest  in  the  technique  and  a  potential  for  wide 
applicability  of  the  CM  focus  groups. 

To  date,  most  studies  comparing  CM  and  FTF  focus  group  performance  have 
centered  only  on  brainstorming  activities  or  asynchronous  discussions  (e.g.,  bulletin 
boards)  rather  than  the  typical  dynamic  group  discussions  associated  with  true  focus 
groups.  Morgan  and  Krueger  (1998)  identify  the  fundamental  aspects  of  a  focus  group  as 
being  research-based,  focused  on  a  topic  of  interest,  and  involving  dynamic  group 
discussion.  The  majority  of  published  studies  comparing  CM  and  FTF  discussions  have 
either  not  been  true  focus  groups  (e.g.,  Barki  &  Pinsonneault,  2001;  Olaniran,  1994; 
Parent,  Gallupe,  Salisbury,  &  Handelman,  2000;  Sweeney,  Soutar,  Hausknecht,  Dallin, 

&  Johnson,  1997)  or  have  not  directly  compared  CM  with  FTF  group  discussions  (e.g., 
Ahern,  1993;  Valacich,  Dennis,  &  Nunamaker,  1992).  A  small  number  of  published 
studies  were  found  that  presented  information  about  actual  CM  focus  group  discussions 
(e.g.,  Schneider,  Kerwin,  Frechtling,  &  Vivari,  2002;  Tse,  1999;  Walston  &  Lissitz, 

2000).  Although  these  studies  reported  results  of  true  CM  focus  groups,  only  Schneider 
et  al.  (2002)  have  actually  compared  the  results  of  FTF  and  CM  groups. 
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Study  Objectives 
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•  Test  a  synchronous  on-line  chat-room  method  of  conducting 
focus  groups 

•  Evaluate  the  equivalence  of  on-line  vs.  face-to-face  modes  of 
conducting  focus  groups 

Equivalence  between  modes  is  essential  in  validating  on-line  focus  groups 
as  an  appropriate  method  of  collecting  data 

•  Assess  the  impact  of  social  presence  on  the  quantity  and  quality 
of  information  gained  from  different  modes  of  conducting  focus 
groups 

•  Collect  content  data  for  Naval  Training  Center  (NTC)  Great 
Lakes 

At  the  request  of  NTC  Great  Lakes,  the  content  of  the  focus  groups  will  be 
Sailor  off-duty  activities 

»  NTC  Great  Lakes  is  interested  in  obtaining  any  information  on  the  off- 
duty  activities  of  their  junior  enlisted  student  population.  Specifically, 
they  are  interested  in  the  most  common  off-duty  activities,  attendance 
at  raves  (all  night  "underground"  parties),  alcohol/drug  use,  and  types 
of  transportation  used 
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A  primary  question  of  researchers  who  utilize  computer-based  focus  groups  is 
whether  the  quantity  or  quality  of  information  from  participants  is  similar  to  traditional 
face-to-face  (FTF)  focus  groups.  Although  Internet-based  focus  groups  may  present  a 
number  of  advantages,  they  are  yet  to  be  proven  as  being  successful  in  capturing  the 
same  amount  or  quality  of  information  typically  obtained  by  traditional  FTF  methods.  A 
secondary  question  is  whether  or  not  differences  in  quantity  or  quality  result  from 
differences  in  perception  or  access  to  social  cues.  The  communication  skills  necessary  to 
participate  in  CM  and  FTF  groups  are  fundamentally  different.  In  FTF  communication, 
people  rely  on  visual,  auditory,  and  verbal  skills  to  interact.  In  a  CM  environment, 
participants  interact  solely  with  a  computer  screen  relying  on  their  ability  to  read, 
comprehend,  and  physically  respond  to  text  conversations  (e.g.,  use  a  mouse  and 
keyboard).  In  a  CM  environment  there  is  less  information  obtained  by  the  participants, 
such  as  social  status  and  body  language.  This  reduction  in  face-to-face  interaction 
introduced  by  CM  focus  groups,  whether  in  the  same  location  or  a  separate  location 
(e.g.,  Internet-based),  may  present  a  potential  threat  to  the  usefulness  and  productivity 
of  the  method.  On  the  other  hand,  they  may  allow  participants  to  express  themselves 
more  freely. 

Since  on-line  or  CM  focus  groups  remain  relatively  untested,  this  study  sought  to 
experimentally  test  one  methodological  approach  to  conducting  these  focus  groups.  To 
meet  this  goal,  the  study  laid  out  a  number  of  objectives.  First,  the  study  sought  to  test  a 
“synchronous”  method  of  conducting  focus  groups  utilizing  a  new  chat  room  format. 
Second,  in  testing  the  new  method  the  study  sought  to  evaluate  the  equivalence  of  on¬ 
line  or  CM  focus  groups  vs.  traditional  FTF  group  discussions.  Third,  in  an  effort  to 
understand  how  differences  in  the  mode  of  participation  might  impact  the  content  of  the 
information  collected,  the  study  sought  to  assess  the  perceptions  of  social  presence  by 
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study  participants.  Fourth,  the  study  sought  to  collect  content  data  for  the  Service 
School  Command  at  Naval  Training  Center  (NTC)  Great  Lakes,  on  the  off-duty  activities 
of  their  junior  enlisted  student  populations.  Specifically,  the  Service  School  Command 
was  interested  in  the  most  common  off-duty  activities  including  attendance  at  raves  (all 
night  “underground”  parties  that  are  often  held  in  abandoned  warehouses  involving 
dancing,  drugs,  and  alcohol),  alcohol/drug  use,  types  of  transportation  being  used,  and 
use  of  on-base  recreational  activities. 
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Sampling  Procedure 


I- 

in 

oc 

CL 

z 


> 

•  Contacted  the  commanding  officer  of  NTC  Great  Lakes  to  secure 
permission  to  conduct  the  study 

Requested  that  participants  be  recruited  from  the  Engineering  and 
Combat  Systems  "A"  School  programs  at  NTC  Great  Lakes 

•  Provided  instructions  on  how  to  recruit  a  volunteer  sample  of 
participants 

-  To  ensure  adequate  numbers  of  participants,  instructions  on 
recruitment  procedures,  an  explanation  of  the  study,  and  sign-up 
sheets  for  participants  were  provided  to  the  point  of  contact  (see 
Appendix  A) 

»  Encouraged  those  recruiting  participants  to  ensure  that 
participants  were  not  forced  but  were  actually  volunteers  for  the 
current  study 

»  Requested  that  NTC  Great  Lakes  recruit  a  total  of  189 
volunteers  (162  for  actual  participation  in  the  study,  as  well  as 
27  alternates) 

Recruitment  of  participants  was  completed  14  days  before 
the  arrival  of  the  research  team  on-site 


22 


The  research  team  partnered  with  the  Service  School  Command  at  NTC  Great  Lakes 
to  collect  both  experimental  data  for  the  project  and  content  data  to  answer  research 
needs  at  the  command.  Permission  was  granted  by  NTC  Great  Lakes  to  the  research 
team  to  recruit  and  conduct  the  study  at  the  Engineering  and  Combat  Systems  “A” 
Schools  located  at  the  command.  To  ensure  that  recruitment  was  conducted 
appropriately,  the  research  team  provided  the  command  with  the  following  information: 
instructions  on  how  to  recruit  participants,  an  explanation  of  the  nature/purpose  of  the 
study,  and  sign-up  sheets  for  the  students.  These  documents  contained  detailed 
instructions  on  how  to  conduct  recruitment  to  ensure  both  voluntary  recruitment  and 
adequate  numbers  of  participants  for  the  study.  The  command  was  instructed  to  recruit 
a  total  of  189  volunteers  for  the  study:  162  Sailors  for  participation  in  the  focus  groups 
and  27  alternates. 

All  procedures  appeared  to  be  followed  as  directed.  Recruitment  of  study 
participants  was  completed  by  the  command  14  days  before  the  research  team  arrived 
on-site  to  conduct  the  study. 
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•  Conducted  focus  groups 

Each  focus  group  was  composed  of  5-6  members  and  a  moderator 

Participant  groups  were  randomly  assigned  to  one  of  three  different  focus 
groups  conditions  (a  total  of  9  focus  groups  were  conducted  in  each  mode) 

Utilized  identical  discussion  protocols  (see  Appendix  B)  for  each  group  that 
focused  on  the  off-duty  recreational  activities  of  participants  (e.g.,  sports, 
parties,  raves,  travel,  transportation,  drug/alcohol  use,  etc.) 

•  On-line  conditions  used  synchronous  chat-room  software 

On-line  focus  groups  were  hosted  on  a  secure  Intranet  local  area  network 
using  standard  Internet  browser  software 

-  Contributions  from  each  participant  were  displayed  in  near  real-time 
Each  comment  submitted  was  clearly  marked  to  indicate  which  participant 
made  the  contribution 

•  Administered  participant  questionnaire  at  the  conclusion  of  the  study 

-  Questionnaire  (see  Appendix  C)  contained  questions  about  participant 
demographics,  experience  with  computers/Internet,  perceptions  of  social 
presence,  introversion,  and  satisfaction  with  the  focus  group  experience 
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Focus  groups  were  conducted  during  the  last  week  of  April  and  first  week  of  May 
2002  at  NTC  Great  Lakes.  All  of  the  focus  groups  were  conducted  in  two  instructional 
classrooms  at  the  command.  A  complete  description  of  each  of  the  conditions  can  be 
found  in  the  next  section;  however,  each  of  the  focus  groups  had  the  following  in 
common:  At  the  start  of  every  group,  participants  as  a  group  were  introduced  to  the 
moderators  and  given  an  explanation  of  the  purpose  of  the  study.  To  ensure  that 
participation  was  voluntary,  participants  were  asked  to  complete  a  short  Informed 
Consent  form  prior  to  beginning  the  study.  After  all  introductions,  completion  of 
consent  forms,  and  answering  of  questions,  participants  were  directed  to  take  a  seat 
either  at  a  computer  workstation  for  computer-based  groups  or  in  one  of  the  chairs  set 
up  in  a  circle  for  face-to-face  group  discussions.  At  this  point,  the  moderator  began  by 
introducing  the  members  of  the  group  and  starting  the  question  protocol.  Each  focus 
group  discussion  was  limited  to  one  hour. 

Participants,  in  groups  of  5-6  were  randomly  assigned  to  one  of  three  conditions 
representing  different  modes  of  communication  with  varying  levels  of  social  interaction. 
The  study  included  traditional  face-to-face  and  two  computer-based  conditions  (i.e., 
computer-mediated  and  Internet-simulated)  in  which  participants  communicated  via 
the  use  of  laptop  computers  and  a  private  Intranet  chat  room.  A  total  of  9  groups  were 
conducted  in  each  of  the  3  conditions,  resulting  in  a  total  of  27  focus  groups.  Focus 
group  conditions  were  balanced  for  time  of  day  and  day  of  week  to  reduce  the  potential 
bias  of  time  in  the  response  of  participant  to  either  the  focus  group  discussion  or  other 
measures. 
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Two  experienced  moderators  led  all  focus  groups,  with  moderators  leading  four  or 
five  focus  groups  in  each  condition.  Following  the  focus  group  discussion,  participants 
were  asked  to  complete  a  short  questionnaire  that  asked  about  their  satisfaction  with  the 
focus  group  experience,  their  use  and  comfort  level  with  computers  for  communication, 
their  perceptions  of  social  presence  conveyed  during  the  discussion,  and  a  few 
demographic  questions. 
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ChatAnywhere©  2.0  software  (LionMax  Software,  2002)  was  used  in  both  computer- 
mediated  and  Internet-simulated  conditions.  The  software  creates  a  chat  room  using 
any  standard  Internet  browser  that  is  connected  to  an  Intranet  or  the  public  Internet. 
Comments  were  posted  in  the  chat  room  after  a  participant  pressed  the  “Enter”  key.  All 
computers  used  in  the  study  were  set  to  refresh  (e.g.,  update)  the  main  screen  once 
every  second  so  that  participants  could  be  assured  of  having  the  most  up-to-date  data. 
Participants  were  able  to  read  all  comments  in  the  order  they  were  posted.  The 
moderator’s  comments  were  always  highlighted  in  yellow  to  help  call  attention  to  their 
questions  and  directions.  Participants  were  instructed  not  to  change  their  font  or 
background  colors  from  black  text  on  a  white  background. 

As  seen  above,  to  access  the  chat  room  participants  used  a  standard  Internet  browser 
that  connected  to  the  chat  room  server  at  a  particular  Internet  Protocol  (IP)  address. 

The  names  of  all  participants  are  listed  in  the  left-hand  column  under  the  title  “on-line 
users.”  Participants  in  the  focus  groups  entered  their  responses  in  the  entry  blank  at  the 
bottom  of  the  screen  and  either  pressed  the  “Enter”  key  on  their  keyboard  or  pressed  the 
“Send”  button  on  the  screen.  The  actual  text  or  response  entered  for  each  participant 
was  displayed  above  the  text  entry  blank  in  order  of  response  submissions.  For  each 
response,  the  name  of  the  participant  was  listed  to  the  left  (including  whom  the  message 
was  addressed  to)  and  the  response  was  displayed  to  the  right.  All  computers  were  set  to 
800  x  600  screen  resolution  to  ensure  that  text  size  was  adequately  large  for  quick 
reading  and  response  by  the  focus  group  participants. 
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Modes  of  Focus  Groups 
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Characteristics: 

•  Same  location 

•  Can  see  and  hear  other 
participants  directly 

•  Interaction  is  direct  via 
face-to-face 
communication 

•  Facilitator  present  in  the 
same  location 


Level  of  Social  Presence: 
HIGH 


Characteristics: 

•  Same  location 

•  Can  see  and  hear  other 
participants  indirectly  (the 
focus  is  on  computer  screen) 

•  Interaction  indirect  via  the 
use  of  computer  keyboard 

•  Facilitator  present  in  the 
same  location 


Level  of  Social  Presence: 
MEDIUM 


Characteristics: 

•  Same  location 

•  Separated  by  barriers  and 
headphones — blocking 
visual  and  auditory  cues 

•  Interaction  indirect  via  the 
use  of  computer  keyboard 

•  Facilitator  present  but 
obscured  by  barriers 


Level  of  Social  Presence: 
LOW 


The  three  focus  group  conditions  used  in  the  study  were  each  designed  to  provide  a 
different  mode  of  communication  and  level  of  social  interaction.  The  first  condition,  the 
traditional  face-to-face  (FTF)  focus  group,  represented  the  most  natural  type  of  group 
discussion.  In  this  condition,  participants  engaged  in  normal  face-to-face 
communication,  utilizing  the  full  spectrum  of  available  verbal  and  non-verbal 
information  on  the  attitudes,  opinions,  and  feelings  of  others  in  the  group.  This 
condition  represented  the  highest  level  of  social  presence  due  to  the  interactive  nature  of 
the  communication  and  the  access  to  the  rich  verbal  and  non-verbal  communication 
from  all  other  participants.  The  content  of  the  discussion  in  the  FTF  focus  groups  was 
audiotaped  and  later  transcribed. 

In  contrast  to  the  traditional  FTF  condition,  the  two  computer-based  conditions 
consisted  of  a  fundamentally  different  form  of  communication.  Participants  in  these 
focus  groups  were  asked  to  use  a  networked  laptop  computer  to  read  and  respond  to  the 
comments  of  other  participants  in  the  discussion.  Participants  in  both  of  these 
conditions  were  told  to  focus  on  what  they  saw  on  the  computer  screen  in  front  of  them, 
and  were  discouraged  from  using  verbal  (e.g.,  talking,  laughing,  sighs,  etc.)  or  non¬ 
verbal  communication  techniques  (e.g.,  gesturing,  facial  expressions,  etc.)  to 
communicate.  ChatAnywhere©  2.0  (LionMax  Software,  2002)  was  used  as  the  software 
platform  to  facilitate  the  focus  group  chat  rooms.  The  software  displayed  the  comments 
of  all  participants  on  the  computer  screen  and  posted  new  comments  in  near  real-time 
(l-second  latency)  after  a  participant  pressed  the  “Enter”  key  on  his  or  her  computer. 
Participants  were  able  to  read  all  comments  in  the  order  they  were  posted  and  respond 
at  any  time.  The  typed  discussion  from  the  computer-mediated  and  Internet-based 
focus  groups  were  saved  and  later  printed. 
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Of  the  two  computer-based  focus  groups,  the  computer-mediated  condition 
represented  an  intermediate  condition  between  the  high  social  presence  setting  of  a 
traditional  FTF  focus  group  and  the  isolation  that  participants  would  experience  if  the 
focus  group  were  taking  place  over  the  Internet.  In  this  condition,  participants  were 
located  in  the  same  room,  but  were  instructed  to  communicate  with  each  other  only  by 
typing  messages  in  the  chat  room  using  their  computer.  However,  while  participants 
communicated  using  their  computers,  they  were  also  able  to  see  and  hear  others  while 
typing  their  responses  into  the  computer. 

The  Internet-simulated  condition  was  designed  to  represent  the  lowest  level  of  social 
presence  by  artificially  separating  participants  from  access  to  any  verbal  or  non-verbal 
communication  cues  while  participating  in  a  chat  room  focus  group  discussion.  In  this 
condition,  participants  were  asked  to  take  part  in  a  discussion  while  being  isolated  from 
one  another  by  physical  barriers  and  the  use  of  noise-reducing  headphones.  This 
condition  represented  the  closest  approximation  available  at  the  data  collection  site  that 
could  simulate  the  isolation  from  verbal,  non-verbal,  or  visual  information  normally 
exchanged  in  social  communication  that  would  be  expected  to  occur  in  a  discussion 
conducted  over  the  Internet. 
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Hypotheses 


On-line  vs.  Face-to-face  (FTF) 
Compared  with  the  FTF  condition, 
participants  in  both  on-line  conditions 
will  demonstrate  a  higher  level  of 
participation  as  measured  by  the  number 
of  comments 

Compared  with  the  FTF  condition, 
participants  in  both  on-line  conditions 
will  provide  more  unique  comments 
The  quality  of  discussion,  as  measured 
by  the  number  relevant  and  off-topic 
comments,  will  be  the  same  for  all 
groups 

Social  Presence 

Compared  with  the  FTF  condition, 
participants  in  both  on-line  conditions 
will  have  lower  perceptions  of  social 
presence 

Participant  satisfaction  with  the  overall 
focus  group  experience  will  increase  as  a 
function  of  the  level  of  social  presence 


Mode  of  Focus  Group 


FTF:  Face-to-face 
CM:  Computer  Mediated 
SI:  Simulated  Internet 


Hypotheses  were  generated  from  a  review  of  the  published  research  literature. 
Hypotheses  were  specified  for  testing  for  differences  between  CM  and  FTF  groups  as 
well  as  the  impact  of  social  presence  on  the  process  of  conducting  the  focus  groups.  The 
following  hypotheses  were  specified  for  CM  and  FTF  group  comparisons  in  the  study: 

l.  Compared  with  the  FTF  condition,  participants  in  both  the  on-line  conditions  will 
demonstrate  a  higher  level  of  participation  as  measured  by  the  number  of 
comments. 


2.  Compared  with  the  FTF  condition,  participants  in  both  on-line  conditions  will 
provide  more  unique  comments. 

3.  The  quality  of  discussion,  as  measured  by  the  number  of  relevant  and  off-topic 
comments,  will  be  the  same  for  all  groups. 

All  hypotheses  were  made  to  be  consistent  with  the  intended  differences  between 
each  of  the  three  conditions.  The  assumption  was  that  the  FTF  condition  would 
represent  the  highest  level  of  social  presence,  with  the  Computer-mediated  being  at  a 
medium  level  of  social  presence  and  the  Internet-simulated  condition  being  the  lowest. 
The  following  hypotheses  were  specified  for  the  impact  of  social  presence  on  group 
comparisons  in  the  study: 


1.  Compared  with  the  FTF  condition,  participants  in  both  on-line  conditions  will 
have  lower  perceptions  of  social  presence. 

2.  Participant  satisfaction  with  the  overall  focus  group  experience  will  increase  as  a 
function  of  the  level  of  social  presence. 
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Data  Processing:  Focus  Group 
Transcripts 

> 

Transcripts  for  all  27  groups  were  subjected  to  independent  content  analysis  by 
4  trained  graduate  students  from  the  University  of  Memphis 

Raters  completed  8  hours  of  training  on  the  coding  methods  used  for  the  study 

»  Training  included  practice  exercises  and  testing  to  ensure  that  consistent 
ratings  were  being  obtained  from  each  of  the  raters. 

•  Ratings  for  7  different  scores  were  recorded  for  each  group  transcript  (see 
Appendix  D): 

Number  of  participation  attempts 

Number  of  relevant  comments 

Number  of  irrelevant  or  off-topic  comments 

Length  of  total  comments 

Length  of  relevant  comments 

Number  of  disagreements 

Total  number  of  comments 

Inter-rater  reliability  assessed  the  agreement  between  graduate  students  in 
their  ratings  of  each  recorded  score 

Inter-rater  reliability  scores  ranged  from  .74-. 99  for  each  of  the  seven  scores 
indicating  adequate  agreement  "between  raters 

The  average  of  the  four  ratings  were  used  as  the  value  for  each  of  the  7  scores 
for  individuals  (and  the  average  of  all  group  members  for  group  level  data)  30 


After  completing  the  data  collection  for  the  study,  transcripts  were  generated  for 
each  of  the  27  focus  groups.  This  involved  transcribing  the  tape  recordings  of  the  focus 
groups  for  all  of  those  groups  in  the  FTF  condition  and  printing  the  computerized  record 
(or  transcript)  for  those  in  the  CM  condition.  Transcripts  and  moderator  notes  for  each 
of  the  27  focus  groups  were  put  into  individual  project  folders  along  with  the 
questionnaires  and  informed  consent  forms  completed  by  each  of  the  participants  in 
that  group. 

One  transcript  from  each  of  the  different  conditions  was  selected  at  random  and 
used  by  the  researchers  to  develop  the  guidelines  for  coding  the  data  from  the  verbatim 
transcripts.  A  guide  for  coding  the  transcripts  was  developed,  along  with  examples  and 
practice  exercises.  These  materials  were  used  to  train  a  team  of  four  graduate  students 
from  The  University  of  Memphis  who  were  hired  to  conduct  the  coding  of  the  data.  The 
graduate  students  received  eight  hours  of  training  in  how  to  code  the  data.  The  training 
included  both  instruction  and  application  exercises  designed  to  ensure  that  all  of  the 
students  were  able  to  consistently  score  the  transcripts  in  a  similar  manner.  All  graduate 
student  raters  were  able  to  successfully  demonstrate  proficiency  in  using  the  scoring 
guidelines  resulting  in  a  high  degree  of  agreement  among  raters. 

In  the  process  of  coding  the  transcripts,  the  raters  recorded  seven  different  pieces  of 
information  on  a  group  rating  form.  A  group  rating  form  was  completed  for  each  of  the 
27  focus  groups  by  each  of  the  4  raters.  The  information  recorded  on  the  group  rating 
form  is  listed  below  with  a  short  description  of  each  of  the  different  scores: 

1.  Number  of  participation  attempts.  Participation  attempts  were  measured  by  the 
number  of  times  an  individual  in  the  group  made  a  comment.  A  participant 
received  credit  for  a  participation  attempt  each  time  they  made  a  comment  in  the 
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conversation.  Multiple  credits  were  not  given  for  a  single  attempt  that  used  more 
than  one  sentence,  unless  it  was  separated  by  comments  from  other  participants. 

2.  Number  of  relevant  comments.  Relevant  comments  were  considered  to  be  ones 
that  were  on  the  topic  of  discussion,  in  reference  to  the  question  being  asked  by 
the  moderator,  or  to  any  previous  question.  Relevant  comments  included 
supporting  examples  and  explanations. 

3.  Number  of  irrelevant  or  off-topic  comments.  Irrelevant  comments  were 
completely  unrelated  to  the  focus  group  topic  or  any  question  posed  by  the 
moderator.  Pleasantries,  small  talk,  comments  about  the  chat  software,  the  room, 
or  the  moderators  themselves  were  also  considered  off-topic. 

4.  Length  of  total  comments.  The  length  of  total  comments  was  measured  by 
counting  the  individual  words  used  in  the  comments  for  each  individual 
participant. 

5.  Length  of  relevant  comments.  The  length  of  relevant  comments  was  measured  by 
counting  the  individual  words  used  in  the  relevant  comments  made  by  individual 
participants  across  the  length  of  each  focus  group. 

6.  Number  of  disagreements.  Disagreements  and  insults  were  recorded  to  measure 
intra-group  conflict.  Disagreements  were  contradictory  statements  made  by  one 
individual  to  another  individual’s  comment.  Insults  were  counted  as  a  conflict  if 
they  were  made  toward  another  participant  in  the  group  or  the  group  in  general. 

7.  Total  number  of  comments.  Total  comments  included  relevant  and  irrelevant 
comments.  Generally,  every  time  a  participant  successfully  entered  the 
conversation  without  being  cut  off,  his  or  her  statement  was  considered  as  one 
comment  regardless  of  the  number  of  ideas  presented  in  that  participation 
attempt. 

Inter-rater  reliability  was  assessed  following  the  study  by  an  audit  of  the  ratings 
generated  by  the  graduate  students.  The  resulting  scores  ranged  from  ,74-.99  for  each 
of  the  7  scores  indicating  adequate  agreement  between  raters.  The  average  of  the  four 
ratings  was  used  as  the  value  for  each  of  the  seven  scores  for  individuals  (and  the 
average  of  all  group  members  for  group  level  data). 
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Data  Processing:  Questionnaire 
Scales 


> 

*  Perceived  Social  Presence  Scale 

Created  an  8-item  scale  to  measure  perceived  social  presence  based  on  the 
published  research  literature  (Rice  1993;  Short  et.  al,  1976;  Tu,  2000) 

I  nternal  reliability  of  the  scale  was  assessed  at  .84  using  the  alpha  coefficient 
A  scale  score  was  created  by  calculating  the  average  for  each  of  the  8  items 
(scores  maintained  their  original  1-5  scale,  with  larger  numbers  meaning  greater 
perceived  social  presence) 

Participant  Satisfaction  Scale 

-  Created  a  4-item  satisfaction  scale  which  focused  on  participants'  enjoyment  of 
the  focus  group  process,  level  of  comfort,  and  willingness  to  participate  in  future 
focus  groups 

I  nternal  reliability  of  the  scale  was  assessed  at  .88  using  the  alpha  coefficient 
A  scale  score  was  created  by  calculating  the  average  for  each  of  the  4  items 
(scores  maintained  their  original  1-5  scale,  with  larger  numbers  meaning  greater 
participant  satisfaction) 

•  Extroversion  Scale 

Utilized  a  20-item  extroversion  scale  from  the  International  Personality  Item  Pool 
(I PIP,  2001)  which  is  based  on  the  Five  Factor  model  of  personality  (McCrae  & 
Costa,  1990). 

I  nternal  reliability  of  the  scale  was  assessed  at  .92  using  the  alpha  coefficient 

-  A  summary  score  was  created  by  adding  the  values  for  each  of  the  20  items 

(resulting  in  a  range  of  scores  20-100,  with  larger  numbers  meaning  greater 
extroversion)  33 


In  addition  to  collecting  data  on  the  focus  group  discussion  via  verbatim  transcripts, 
the  study  also  required  participants  to  complete  a  short  4-page  questionnaire  and  an 
Informed  Consent  form).  The  questionnaire  assessed  participant  background 
characteristics,  extroversion  personality  characteristics,  and  perceptions  of  the  focus 
group  experience.  The  questionnaire  contained  a  description  of  the  study,  Privacy  Act 
statement,  and  questions  that  were  divided  into  two  sections,  “Your  Background”  and 
“Focus  Group  Evaluation.”  The  section  titled  “Your  Background”  focused  on  assessing 
demographic  characteristics,  and  experience/comfort  with  the  use  of  computers  and  the 
Internet  for  communication  activities  such  as  e-mail,  on-line  chats,  instant  messaging, 
or  others.  The  section  titled  “Focus  Group  Evaluation”  contained  questions  focused  on 
assessing  participant  satisfaction,  perceptions  of  social  presence,  and  extroversion 
personality  tendencies.  Scores  for  each  of  these  constructs  were  computed  and  used  in 
the  study  as  variables  in  the  analyses.  A  description  of  each  of  these  scores  and  how  they 
were  computed  can  be  found  below: 

1.  Perceived  Social  Presence.  Since  there  was  no  agreed  upon  measure  of  social 
presence  in  the  literature,  the  authors  constructed  a  scale  containing  18  items 
that  assessed  participant  perceptions  of  various  attributes  of  social  presence 
experienced.  Items  for  this  scale  were  taken  from  the  work  of  Rice  (1993),  Short 
et  al.  (1976),  and  Tu  (2000).  The  questions  in  this  scale  were  presented  as 
statements  that  study  participants  rated  according  to  how  much  they  agreed  or 
disagreed  using  a  5-point  Likert-type  scale.  To  assess  how  well  the  questions  held 
together  as  a  scale,  they  were  subjected  to  internal  reliability  analysis  by  use  of 
the  coefficient  alpha  method.  Initial  reliability  estimates  indicated  the  need  to 
drop  five  questions  from  the  scale  to  improve  internal  consistency.  Dropping 
these  poor  items  resulted  in  a  coefficient  alpha  of  .84,  indicating  that  the  scale 
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had  high  consistency.  A  scale  score  for  social  presence  was  created  by  calculating 
the  average  of  responses  to  the  13  remaining  items  for  each  participant.  The 
resulting  scores  maintained  the  original  range  of  1-5,  with  larger  numbers 
indicating  greater  perceived  social  presence. 

2.  Participant  Satisfaction.  The  authors  wrote  a  short  scale  of  six  questions  to 
assess  global  perceptions  of  the  focus  group  experience.  The  questions  were 
presented  as  statements  that  study  participants  rated  how  much  they  agreed  or 
disagreed  using  a  5-point  Likert-type  scale.  Within  this  group  of  questions,  four 
items  focused  on  participant  satisfaction  with  the  focus  group  experience  and 
their  willingness  to  take  part  in  future  focus  groups.  The  questions  were 
subjected  to  internal  reliability  analysis  by  use  of  the  coefficient  alpha  method. 
The  resulting  coefficient  alpha  of  .88  indicated  that  the  scale  had  high 
consistency.  Calculating  the  average  of  responses  to  the  four  items  for  each 
participant  created  a  scale  score  for  participant  satisfaction.  The  resulting  scores 
maintained  the  original  range  of  1-5,  with  larger  numbers  indicating  greater 
satisfaction. 

3.  Extroversion  Tendencies.  To  measure  the  level  of  extroversion  personality 
tendencies  present  in  study  participants,  the  researchers  used  a  20-item  scale 
from  the  International  Personality  Item  Pool  (IPIP,  2001).  The  measure  is  a 
statistically  equivalent  alternative  developed  by  IPIP  as  a  measure  of  extroversion 
as  conceived  by  the  Five  Factor  model  of  personality  (McCrae  &  Costa,  1990).  The 
questions  in  this  scale  were  presented  as  statements  that  study  participants  rated 
according  to  how  much  they  agreed  or  disagreed  using  a  5-point  Likert-type 
scale.  The  questions  were  subjected  to  internal  reliability  analysis.  The  resulting 
coefficient  alpha  of  .92,  which  is  consistent  with  previous  IPIP  research  (IPIP, 
2000),  indicated  that  the  scale  had  high  consistency.  A  summary  score  was 
created  using  the  20  items  by  adding  the  values  of  each  item  together.  The 
resulting  scores  ranged  from  20-100,  with  larger  numbers  indicating  greater 
extroversion  tendencies. 
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The  following  section  provides  a  review  of  the  results  of  the  study.  A  short 
description  of  the  characteristics  of  the  sample  appears  first,  followed  by  a  presentation 
of  the  results  in  order  of  the  hypotheses  stated  in  the  methods  section  of  this  report.  In 
addition  to  the  seven  stated  hypotheses,  two  additional  auxiliary  hypotheses  were  tested 
and  their  results  appear  in  this  report.  The  first  of  these  auxiliary  hypotheses  evaluates 
the  differences  in  the  level  of  intra-group  conflict  experienced  in  each  of  the  three  study 
conditions.  Finally,  this  section  concludes  with  a  simple  listing  of  the  most  significant 
findings  or  themes  from  the  content  analyses  of  the  verbatim  transcripts  of  the  focus 
groups.  This  information  is  provided  to  give  the  reader  an  idea  of  the  practical  value  of 
conducting  the  on-line  focus  groups  to  gain  content  information  about  a  particular 
topic. 
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Participant  Characteristics 
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•  Participants  in  the  study 
were  generally  young, 
junior  enlisted,  and  male. 

Age 

»  Range:  18-37  years 
»  Average:  21.4  years 
Paygrade 

»  Range:  E-l  to  E-5 
»  Average:  E-2 
Gender 

»  Male:  82%  (N  =  127) 

>»  Female:  18%  (N  =  27) 


■  E-l  □  E-2  ■  E-3  B  E-4  or  above 
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A  total  of  156  junior  enlisted  Sailors  (E-l  to  E-3)  and  petty  officers  (E-4  to  E-6)  from 
NTC  Great  Lakes  participated  in  the  study.  In  two  cases  participants  did  not  respond  to 
all  of  the  demographics  leading  to  totals  on  most  demographics  of  154.  In  general, 
participants  were  young,  in  the  lower  paygrades,  and  male.  Participants  averaged  21 
years  of  age,  with  a  range  in  age  from  18  to  37  years.  Only  18  percent  of  the  participants 
in  the  study  were  female,  which  is  slightly  higher  than  the  current  gender  distribution  of 
Sailors  in  the  Navy.  All  participants  in  the  focus  groups  were  lower  ranking  enlisted 
Sailors  who  were  then  attending  “A”  School  programs  at  the  Service  School  Command  at 
NTC  Great  Lakes.  “A”  School  programs  represent  the  initial  occupational  specialty 
training  obtained  by  enlisted  Sailors  at  the  start  of  their  military  career.  Since  junior 
enlisted  Sailors  make  up  the  majority  of  the  personnel  population  at  the  base,  they  were 
targeted  as  participants  for  the  study.  In  the  present  study,  93  percent  of  those  who 
participated  in  the  focus  groups  were  in  the  category  of  junior  enlisted  personnel. 


30 


I- 

in 

cc 

CL 


Participant  Experience  with 
Computers/Internet 


•  Study  participants  had  a 
great  deal  of  experience  and 
comfort  in  working  with 
computers 

-  92%  had  experience  with 
computers 

-  89%  reported  that  they  were 
very  comfortable  working  with 
computers 

-  93%  have  used  the  Internet  at 
some  time 

»  The  majority  of  participants 
(51% )  had  4  or  more  years 
of  experience  with  using  the 
I  nternet  at  the  time  of  the 
study 

83%  had  used  Internet  chat  or 
instant  messaging  software 
83%  reported  that  they  were 
comfortable  using  Internet  chat 
or  instant  messaging  software 


Frequency  of  Internet  Use 

50  t - 


45 

40 


■  Never 

■  Once  a  week 

■  Several  times  a  week 

■  Once  a  day 

U  Several  times  a  day 
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Participants  in  the  study  had  a  high  degree  of  experience  and  familiarity  with  using 
computers  and  communicating  on  the  Internet.  Ninety-two  percent  reported  that  they 
had  previous  experience  with  computers,  and  most  (89%)  were  comfortable  with  using 
computers  in  a  wide  variety  of  contexts.  In  addition,  participants  had  a  great  deal  of 
familiarity  with  the  Internet.  Ninety-three  percent  of  participants  had  used  the  Internet 
at  some  time  in  the  past  five  years.  Participants  showed  a  great  deal  of  experience  with 
the  Internet,  with  the  majority  (51%)  reporting  four  or  more  years  of  active  use  of  the 
Internet.  Eighty-three  percent  of  the  enlisted  Sailor  participants  had  previous 
experience  with  chat  rooms  or  instant  messaging  over  the  Internet.  Of  this  group,  all 
participants  reported  that  they  were  comfortable  with  this  style  of  communication  and 
that  they  frequently  used  chat  rooms  when  given  the  opportunity. 

In  contrast  to  the  great  deal  of  experience  participants  had  with  the  Internet,  when 
asked  how  often  participants  used  the  Internet  since  they  joined  the  Navy,  over  half 
reported  that  they  have  used  the  Internet  less  than  once  a  week.  The  primary  reason 
given  for  this  by  study  participants  was  that  they  did  not  have  good  access  on  base  to  the 
Internet.  Study  participants  reported  that  as  students,  there  were  only  a  few  locations  on 
base  for  them  to  gain  access  to  the  Internet,  that  the  majority  charged  for  access  by  the 
minute,  and  that  all  of  these  locations  tended  to  have  long  lines  of  Sailors  waiting  to  use 
the  computers.  Most  of  the  participants  reported  that  they  could  not  afford  to  pay  the 
fees  and  so  they  did  not  use  the  Internet  as  regularly  as  they  had  prior  to  joining  the 
Navy. 
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Level  of  Participation 
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•  Hypothesis  1: 

-  Compared  with  the  face-to- 
face  condition,  participants  in 
both  on-line  conditions  will 
demonstrate  a  higher  level  of 
participation  as  measured  by 
the  number  of  comments 

•  Results: 

Individual  Participation  Rate: 

»  Main  Effect:  Non¬ 
significant 

F  (2,  153)  =  0.45,  p  = 
0.98 

»  No  difference  was  found 
in  the  number  of 
comments  made  by 
participants  in  each  of  the 
different  modes 


> 

Individual  Participation  Rates  by 
Type  of  Focus  Group 


M 

SD 

n 

Internet-based 

66.8 

32.5 

51 

Computer-mediated 

68.5 

23.6 

52 

Face-to-face 

63.4 

27.8 

53 
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The  first  hypothesis  stated  that  “compared  with  the  face-to-face  condition, 
participants  in  both  the  on-line  conditions  will  demonstrate  a  higher  level  of 
participation  as  measured  by  the  number  of  comments.”  Individual  participation  rates 
were  calculated  by  counting  the  number  of  comments  each  participant  made  during  the 
process  of  the  focus  group  discussion.  Since  the  focus  of  the  hypothesis  was  on  the 
impact  of  the  conditions  on  individuals,  the  participation  rates  were  calculated  and 
tested  at  the  individual  level.  The  means  obtained  from  individual  participation  rates 
across  the  three  conditions  were  similar;  however,  as  predicted,  the  computer-based 
conditions  had  higher  mean  scores  than  the  face-to-face  condition,  but  they  were  not 
significantly  different.  The  means  were  subjected  to  an  analysis  of  variance  (ANOVA) 
test,  which  showed  no  statistically  significant  differences  in  the  individual  level  of 
participation  across  the  three  conditions,  F  (2, 153)  =  0.45,  p  =  0.98. 
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Quantity  of  Information  Obtained 
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•  Hypothesis  2: 

-  Compared  with  the  face-to-face 
condition,  participants  in  both  on¬ 
line  conditions  will  provide  more 
unique  comments 

•  Results: 

Number  of  Unique  Ideas 
Generated: 

»  Main  Effect:  Non-Significant 

-  F  (2,  24)  =  1,35,  p  =  0.28 
»  No  difference  was  found  in  the 

number  of  unique  ideas 
generated  by  groups  in  each  of 
the  different  modes 

Average  Number  of  Words  Used: 

»  Main  Effect:  Significant 

-  F  (2,  24)  =  47.33,  p  <  .01 
»  Participants  in  the  face-to-face 

condition  used  a  significantly 
greater  number  of  words  while 
participating  in  the  discussion 


Number  of  Unique  Ideas  Generated 
by  Type  of  Focus  Group 


M 

SD 

n 

Internet-based 

55.2 

11.4 

9 

Computer-mediated 

48.4 

12.7 

9 

Face-to-face 

54.1 

13.6 

9 

Number  of  Words  Used  by  Type  of 
Focus  Group 


M 

SD 

n 

Internet-based 

2,361 

658.7 

9 

Computer-mediated 

2,593 

877.0 

9 

Face-to-face 

6,234 

1,219.7 

9 
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The  second  hypothesis  stated  that  “compared  with  the  face-to-face  condition, 
participants  in  both  on-line  conditions  will  provide  more  unique  comments.”  Evaluation 
of  this  hypothesis  focused  on  two  types  of  information  obtained  from  participants:  the 
number  of  unique  ideas  generated  and  the  overall  number  of  words  used  in  focus 
groups.  The  first  type  of  information,  number  of  unique  ideas  generated,  represents  the 
most  significant  part  of  trying  to  answer  this  question.  To  remain  consistent  with 
previous  studies  in  the  literature  (e.g.,  Bordia,  1997;  Olaniran,  1994;  Parent  et  al., 

2000),  the  quantity  of  information  produced  by  each  group  in  the  three  conditions  was 
measured  by  total  number  of  unique  ideas.  This  is  primarily  due  to  the  fact  that  one  of 
the  main  reasons  for  conducting  focus  groups  is  to  obtain  unique  and  explanatory 
content  or  qualitative  data  on  a  particular  subject.  A  related,  but  much  less  significant, 
measure  of  the  quantity  of  information  obtained  in  a  focus  group  is  the  sheer  number  of 
words  used  to  communicate  in  the  discussion  process.  This  measure  may  be  problematic 
due  to  the  fact  that  participants  may  say  a  great  deal  without  generating  useful  or 
unique  opinions,  reactions,  or  ideas  during  the  discussion.  This  second  type  of 
information  is  evaluated  here  as  a  way  of  providing  a  more  comprehensive  evaluation  of 
the  overall  quantity  of  information  generated  during  the  course  of  the  focus  group 
conducted  in  this  study. 

The  number  of  unique  ideas  generated  by  each  focus  group  was  calculated  by 
summing  the  number  of  unique  comments  made  by  all  individuals  in  each  group.  The 
means  were  subjected  to  an  ANOVA  test,  which  showed  no  significant  differences  in  the 
number  of  unique  ideas  across  the  three  conditions,  F  (2,  24)  =  1.35,  p  =  0.28.  So  while 
the  individual  means  appeared  to  show  some  variation,  no  differences  were  found 
between  them  on  the  number  of  unique  ideas  generated. 
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The  average  number  of  words  used  by  participants  to  communicate  during  the  focus 
group  was  calculated  by  summing  the  number  of  words  by  each  individual  within  a 
group.  This  number  was  then  divided  by  the  actual  number  of  participants  in  the  group 
to  arrive  at  the  average  number  for  the  group.  The  means  were  subjected  to  an  ANOVA 
test,  which  showed  a  significant  difference  in  the  number  of  words  used  across  the  three 
conditions,  F  (2,  24)  =  49.60,  p  <  0.01.  The  results  indicated  that  those  in  the  face-to- 
face  conditions  used  significantly  more  words  than  those  in  either  of  the  computer- 
based  conditions.  This  finding  is  not  surprising;  however,  in  that  it  is  well  known  that 
people  tend  to  use  more  words  when  communicating  verbally  than  in  writing. 

Together  these  results  indicate  that  while  participants  used  more  words  in  the  face- 
to-face  conditions,  all  groups  tended  to  produce  similar  results  in  the  overall  number  of 
unique  ideas  generated  during  the  discussion.  Both  face-to-face  and  computer-based 
focus  groups  tend  to  produce  similar  amounts  of  information,  even  though  participants 
use  fewer  words  in  computer-based  conditions. 
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Quality  of  Information  Obtained 
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Hypothesis  3: 

The  quality  of  discussion,  as 
measured  by  the  number  of 
relevant  and  off-topic  comments, 
will  be  the  same  for  all  groups 
Results: 

Number  of  Relevant  Comments: 

»  Main  Effect:  Non-Significant 

-  F  (2,  24)  =  0.76,  p  =  0.48 

»  No  difference  was  found  in  the 
number  of  relevant  comments 
made  by  groups  in  each  of 
different  modes 

Number  of  Off-topic  Comments: 

»  Main  Effect:  Significant 

-  F  (2,  24)  =  8.56,  p  <  0.01 

»  Participants  in  the  Computer- 
Mediated  and  Internet 
Simulated  focus  group 
conditions  were  more  likely  to 
make  off-topic  comments  while 
participating  in  the  discussion 


Number  of  Relevant  Comments  by 
Type  of  Focus  Group 


M 

SD 

n 

Internet-based 

309.8 

111.5 

9 

Computer-mediated 

308.4 

81.0 

9 

Face-to-face 

356.1 

84.6 

9 

Number  of  Off-topic  Comments  by 
Type  of  Focus  Group 


M 

SD 

n 

Internet-based 

68.6 

22.8 

9 

Computer-mediated 

87.4 

58.3 

9 

Face-to-face 

17.1 

15.9 

9 
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The  third  hypothesis  stated  that  “the  quality  of  discussion,  as  measured  by  the 
number  of  relevant  and  off-topic  comments,  will  be  the  same  for  all  groups.”  The  focus 
of  this  hypothesis  was  on  how  well  the  participants  in  the  three  conditions  were  able  to 
stay  on  task  in  the  focus  group  discussions.  Groups  that  are  able  to  stay  on  task  and 
focus  on  the  topic  of  discussion  have  been  found  to  generate  more  useful  data  than 
groups  that  frequently  stray  away  from  the  topic  of  focus  (Edmunds,  1999).  To  evaluate 
this  hypothesis,  two  opposing  types  of  information  were  collected  and  tested  for 
differences:  the  number  of  relevant  comments  and  the  number  of  off-topic  comments. 
The  number  of  relevant  comments  was  used  to  assess  the  usefulness  of  the  information 
obtained.  Groups  that  produce  more  relevant  or  on-topic  comments  should  yield  better 
data.  In  contrast,  the  number  of  off-topic  comments  represents  the  amount  of  time  that 
the  group  members  were  engaging  in  communication  unrelated  to  the  purpose  of  the 
group.  The  higher  the  number  of  off-topic  comments  in  a  group,  the  more  difficulty  a 
focus  group  will  have  in  producing  useful  data.  The  present  study  assumed  that  there 
would  be  no  difference  between  the  numbers  of  on-topic  (i.e.,  relevant)  and  off-topic 
(i.e.,  irrelevant)  comments  made  by  participants  in  the  three  conditions. 

The  number  of  on-topic  comments  was  calculated  by  summing  the  number  of 
comments  made  by  each  individual  that  were  on  the  topics  or  questions  introduced  by 
the  moderator.  The  numbers  for  each  individual  were  then  added  together  and  divided 
by  the  actual  number  of  participants  in  that  group  to  arrive  at  an  average  number  of  on- 
topic  words  used  in  the  discussion.  The  means  were  subjected  to  an  ANOVA  test,  which 
showed  no  significant  differences  in  the  number  of  relevant  comments  made  by  group 
members  in  each  of  the  three  conditions,  F  (2,  24)  =  0.76,  p  =  0.48. 
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The  number  of  off-topic  comments  was  calculated  in  a  similar  manner  to  the  on- 
topic  comments;  however,  the  focus  was  on  the  number  of  comments  that  were  made 
about  topics  other  than  what  had  been  introduced  by  the  moderator.  The  means  were 
subjected  to  an  ANOVA  test,  which  showed  a  significant  difference  in  the  number  of  off- 
topic  comments  made  by  participants  across  the  three  conditions,  F  (2,  24)  =  8.56, 
p  <  0.01.  Post-hoc  Tukey  tests  were  conducted  to  evaluate  which  conditions  differed 
from  the  others.  The  results  indicated  that  both  of  the  computer-based  conditions  were 
significantly  more  likely  to  be  off-topic  than  participants  in  the  face-to-face  condition. 
Together,  the  results  indicate  that  even  though  the  discussion  in  both  computer- 
mediated  and  Internet-based  conditions  got  off-topic  more  than  face-to-face  groups, 
they  still  produced  similar  numbers  of  unique  ideas  and  relevant  comments. 
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Perceived  Social  Presence 
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•  Hypothesis  4: 

-  Compared  with  the  face-to- 
face  condition,  participants  in 
both  on-line  conditions  will 
have  lower  perceptions  of 
social  presence 

•  Results: 

Perceived  Social  Presence: 


»  Main  Effect:  Non¬ 
significant 

F  (2,  152)  =  0.56,  p  = 
.57 

»  No  difference  was  found 
in  the  perceived  level  of 
social  presence  by 
participants  in  each  of  the 
different  modes 


Perceived  Social  Presence  by  Type 
of  Focus  Group 


M 

SD 

n 

Internet-based 

4.2 

0.48 

50 

Computer-mediated 

4.1 

0.56 

52 

Face-to-face 

4.2 

0.39 

53 
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The  fourth  hypothesis  stated  that  “compared  with  the  face-to-face  condition, 
participants  in  both  on-line  conditions  will  have  lower  perceptions  of  social  presence.” 
The  focus  of  this  hypothesis  was  on  the  level  of  social  presence  perceived  by  each  of  the 
participants  in  the  three  conditions  of  the  focus  group  discussions.  The  means  of  the 
scores  for  the  three  groups  appeared  to  be  almost  identical  with  the  overall  average 
being  4.2  (a  moderately  high  level  of  social  presence)  across  the  3  conditions.  An 
ANOVA  test  showed  no  significant  differences  in  the  perceived  level  of  social  presence 
as  reported  by  group  members  in  each  of  the  three  conditions,  F  (2, 152)  =  0.56,  p= 
0-57- 
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Participant  Satisfaction 
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Hypothesis  5: 

Participant  satisfaction  with  the 
overall  focus  group  experience  will 
increase  as  a  function  of  the  level 
of  social  presence 

Results: 

-  Correlation  of  Perceived  Social 
Presence  and  Participant 
Satisfaction 

»  Main  Effect:  Significant 

-  r  =  .51  (p  <  0.01,  n  =  152) 

»  Significant  correlation  was  found 
between  perceived  social 
presence  and  participant 
satisfaction 

Participant  Satisfaction: 

»  Main  Effect:  Nonsignificant 

-  F  (2,  150)  =  0.04,  p  >  .96 

»  No  difference  was  found  in  the 
level  of  participant  satisfaction 
in  each  of  the  different  modes 
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Computer-mediated 

4.0 
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Face-to-face 

4.0 

0.54 
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The  fifth  hypothesis  stated  that  “participant  satisfaction  with  the  overall  focus  group 
experience  will  increase  as  a  function  of  the  level  of  social  presence.”  The  focus  of  this 
hypothesis  was  on  the  level  of  satisfaction  with  the  focus  group  reported  by  members  of 
the  groups  in  each  of  the  three  conditions.  The  means  for  participant  satisfaction  did  not 
appear  to  vary  among  participants,  with  87  percent  being  satisfied  with  their  experience 
in  the  focus  group.  To  test  the  fifth  hypothesis,  the  relationship  between  perceived  social 
presence  and  participant  satisfaction  was  first  explored  by  use  of  a  Pearson  product 
moment  correlation  coefficient.  The  resulting  Pearson  product  moment  correlation 
coefficient  indicated  that  there  was  a  significant  relationship  between  perceived  social 
presence  and  participant  satisfaction,  r  =  .51  (p  <  0.01,  n  =  152).  In  addition,  an  ANOVA 
showed  no  significant  differences  in  the  level  of  satisfaction  by  group  members  in  each 
of  the  three  conditions,  F  (2, 150)  =  0.04,  p  =  0.96. 

Taken  together,  these  results  indicate  that  while  there  is  a  relationship  between 
perceived  social  presence  and  participant  satisfaction,  no  actual  group  differences  were 
found  in  the  level  of  participant  satisfaction.  It  does  not  appear  that  the  focus  group 
condition  contributed  significantly  to  ratings  of  satisfaction  with  the  focus  group 
experience. 
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Intra-group  Conflict 
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•  Auxiliary  Hypothesis  1: 

The  amount  of  intra-group 
conflict  (e.g.,  disagreements, 
insults,  or  other  conflicts)  will  be 
the  same  for  all  participation 
modes 

•  Results: 

Intra-group  Conflict: 

»  Main  Effect:  Significant 
-  F  (2,  24)  =  5.64,  p  = 

0.01 

»  Participants  in  the  computer- 
mediated  focus  group 
condition  were  significantly 
more  likely  to  make  insults  or 
have  disagreements  with 
other  participants  than  those 
in  other  modes  of 
participation 


Number  of  Disagreements,  Insults, 
or  Other  Conflicts  by  Type  of 
Focus  Group 
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Face-to-face 
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The  fifth  hypothesis  stated  that  “participant  satisfaction  with  the  overall  focus  group 
experience  will  increase  as  a  function  of  the  level  of  social  presence.”  The  focus  of  this 
hypothesis  was  on  the  level  of  satisfaction  with  the  focus  group  reported  by  members  of 
the  groups  in  each  of  the  three  conditions.  The  means  for  participant  satisfaction  did  not 
appear  to  vary  among  participants,  with  87  percent  being  satisfied  with  their  experience 
in  the  focus  group.  To  test  the  fifth  hypothesis,  the  relationship  between  perceived  social 
presence  and  participant  satisfaction  was  first  explored  by  use  of  a  Pearson  product 
moment  correlation  coefficient.  The  resulting  Pearson  product  moment  correlation 
coefficient  indicated  that  there  was  a  significant  relationship  between  perceived  social 
presence  and  participant  satisfaction,  r  =  .51  (p  <  0.01,  n  =  152).  In  addition,  an  ANOVA 
showed  no  significant  differences  in  the  level  of  satisfaction  by  group  members  in  each 
of  the  three  conditions,  F  (2, 150)  =  0.04,  p  =  0.96. 

Taken  together,  these  results  indicate  that  while  there  is  a  relationship  between 
perceived  social  presence  and  participant  satisfaction,  no  actual  group  differences  were 
found  in  the  level  of  participant  satisfaction.  It  does  not  appear  that  the  focus  group 
condition  contributed  significantly  to  ratings  of  satisfaction  with  the  focus  group 
experience. 

For  the  purposes  of  measuring  intra-group  conflict  in  this  study,  the  researchers 
constructed  a  measure  of  conflict  by  having  the  graduate  students  code  the  number  of 
disagreements  that  occurred  in  the  process  of  conducting  each  focus  group. 
Disagreements  between  group  members,  insults  directed  at  other  group  members,  and 
other  contradictory  statements  made  during  the  course  of  the  focus  group  were  all 
coded  as  conflict  in  this  study.  Scores  for  each  individual  were  constructed  by  counting 
the  number  of  statements  that  were  coded  as  a  disagreement  or  conflict.  To  construct 


39 


the  group  level  measure  of  conflict,  scores  for  each  of  the  participants  were  summed  to 
arrive  at  a  total  number  of  disagreements  or  other  conflicts  experienced  by  the  group. 
An  ANOVA  test  showed  significant  differences  in  the  amount  of  conflict  between  group 
members  in  each  of  the  three  conditions,  F  (2,  24)  =  5.64,  p  =  0.01. 

Post-hoc  Tukey  tests  were  conducted  to  evaluate  which  conditions  differed  from  the 
others.  Participants  in  the  computer-mediated  condition  showed  significantly  more 
conflict  or  disagreements  with  other  group  members  than  those  in  either  of  the  other 
two  conditions.  While  this  finding  may  be  surprising,  the  condition  presented  an 
interesting  combination  of  features  of  the  other  two  conditions.  While  participants  in 
this  mode  contributed  to  the  focus  group  discussion  via  their  computers,  they  had  full 
access  to  visual,  auditory,  and  other  social  cues  (e.g.,  they  were  able  to  look  up  from 
their  computer  to  see  and  hear  others).  Being  able  to  see  and  hear  other  participants 
may  have  contributed  in  some  way  to  the  observed  higher  level  of  conflict. 
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•  Auxiliary  Hypothesis  2: 

Satisfaction  with  the  focus  group 
experience  will  increase  as  a  function  of 
extroversion 

•  Results: 

-  Level  of  Extroversion 

»  Main  Effect:  Nonsignificant 

-  F  (2,  150)  =  0.95,  p  =  0.42 

»  No  differences  were  found  in  the  level 
of  extroversion  in  each  of  the 
different  modes 

-  Correlation  of  Level  of  Extroversion  with 
Participant  Satisfaction 

»  Main  Effect:  Significant 

-  r  =  .21  (p  <  0.01,  n  =  153) 

»  Significant  correlation  was  found 
between  extroversion  and  satisfaction 

Participant  Satisfaction  using  Extroversion 
as  a  Covariate 

»  Main  Effect:  Nonsignificant 

-  F  (2,  149)  =  0.10,  p  =  0.41 


»  No  differences  were  found  between 
satisfaction  in  each  of  different  modes 


Level  of  Extroversion  by  Type  of 
Focus  Group 
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The  second  auxiliary  hypothesis  stated  that  “participant  satisfaction  with  the  overall 
focus  group  experience  will  increase  as  a  function  of  extroversion.”  This  hypothesis 
focused  on  the  impact  of  extroversion,  a  personality  characteristic  that  drives  people 
toward  social  contact,  on  the  focus  group  process.  First,  the  researchers  were  interested 
in  any  potential  differences  in  the  level  of  extroversion  found  between  the  participants 
in  each  of  the  conditions.  This  was  important  to  establish  that  there  was  no  systematic 
bias  being  introduced  by  extroversion  which  had  been  disproportionately  distributed  to 
one  or  another  group  (or  condition).  Second,  the  researchers  were  interested  in  the 
relationship  between  extroversion  and  satisfaction  with  the  focus  group  process. 
Previous  research  by  McCrae  and  Costa  (1990)  has  shown  that  those  high  in 
extroversion  tend  to  prefer  higher  levels  of  social  interaction  than  those  low  in  the 
personality  characteristic.  Specifically,  the  researchers  proposed  that  as  the  level  of 
social  interaction  increased,  those  high  in  extroversion  would  be  more  satisfied  with  the 
focus  group  experience  than  those  low  in  extroversion. 

For  the  purposes  of  measuring  extroversion,  the  researchers  used  a  20-item  measure 
developed  by  the  International  Personality  Item  Pool  (2001).  To  evaluate  the  first  part 
of  this  hypothesis,  the  mean  scores  were  subjected  to  an  ANOVA  test,  which  showed  no 
significant  differences  in  the  level  of  extroversion  between  group  members  in  each  of  the 
three  conditions,  F  (2, 150)  =  0.95,  p  =  0.42.  To  answer  the  second  part  of  the 
hypothesis,  the  researchers  evaluated  the  relationship  between  level  of  extroversion  and 
participant  satisfaction  by  use  of  a  Pearson  product  moment  correlation  coefficient.  The 
resulting  Pearson  product  moment  correlation  coefficient  indicated  that  there  was  a 
significant  relationship  between  perceived  extroversion  and  participant  satisfaction, 
r  =  .21  (p  <  0.01,  n  =  153).  Since  extroversion  was  shown  to  be  significantly  correlated 
with  participant  satisfaction,  an  additional  ANOVA  test  was  conducted  to  test  for  group 
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differences  using  extroversion  as  a  covariate.  This  analysis  showed  no  significant 
differences  in  the  level  of  satisfaction  by  group  members  in  each  of  the  three  conditions, 
F  (2, 150)  =  0.10,  p  =  041. 

Taken  together,  these  results  indicate  that  while  extroversion  and  participant 
satisfaction  are  related,  extroversion  does  not  appear  to  cause  differences  in  the  level  of 
satisfaction  reported  by  participants. 
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Major  Content  Findings 
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•  Few  junior  enlisted  Sailors  report  attending  raves  or  know  others  who 
attended  rave  parties 

-  Most  participants  agreed  that  raves  were  not  the  kind  of  event  they  either 
wanted  to  attend  or  be  caught  attending  by  superiors 

-  Sailors  who  had  attended  raves  in  the  past  reported  that  the  music  and  dancing 
rather  than  the  drugs  were  the  most  important  reason  they  attended  these 
events 

•  Obtaining  cost-effective  and  safe  transportation  appeared  to  be  the  major 
concern  of  junior  enlisted  Sailors  with  respect  to  their  off-duty  activities 

The  lack  of  dependable  transportation  on  base  was  often  cited  as  a  major 
problem  for  junior  enlisted  Sailors  who  have  limited  time  off  from  their  duties 
The  most  common  off-base  transportation  used  by  Sailors  were  the  "Jam  Vans" 
»  This  mode  of  transportation  was  cited  as  being  expensive,  unreliable  (e.g., 
drivers  apparently  do  not  always  come  back  and  pick  up  passengers  after 
dropping  them  off)  and  potentially  dangerous  (e.g.,  poor  driving  habits, 
drugs/alcohol  present  at  times,  etc.) 

•  Sailors  overwhelmingly  preferred  to  be  off  the  base  whenever  possible  for 
their  off-duty  activities 

The  most  common  activities  cited  by  junior  enlisted  Sailors  were  going  to  local 
shopping  malls,  visiting  the  city  of  Chicago,  or  visiting  friends/family  in  the  area 
around  fne  base 
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This  slide  will  deal  with  a  short  review  of  the  major  themes  in  the  content  data 
generated  in  the  focus  groups.  This  involves  careful  reading  of  each  of  the  transcripts 
and  recording  of  the  number  of  times  a  particular  response  is  made  or  any  similar 
responses.  The  topic  of  the  focus  groups  for  this  study  centered  on  the  off-duty  activities 
of  Sailors  stationed  at  NTC  Great  Lakes.  Discussions  included  both  on-base  and  off-base 
activities,  transportation,  costs,  and  a  special  focus  on  raves. 

Contrary  to  what  officials  at  NTC  Great  Lakes  suspected,  the  content  of  the  group 
discussions  indicated  that  few  junior  enlisted  Sailors  reported  attending  raves  or  knew 
others  who  attended  rave  parties.  Most  of  the  participants  agreed  that  raves  were  not 
the  kind  of  event  they  either  wanted  to  attend  or  be  caught  attending  by  superiors.  Many 
indicated  that  they  were  instructed  by  their  teachers  and  other  members  of  the  staff  to 
avoid  raves  due  to  the  negative  impact  that  an  arrest  at  one  of  these  parties  might  have 
on  their  Navy  career.  The  few  Sailors  who  reported  that  they  had  attended  raves  in  the 
past  said  that  the  music  and  dancing  rather  than  the  drugs  were  the  most  important 
reason  they  attended  these  events.  However,  the  authors  suggest  caution  in  interpreting 
these  content  findings.  Participation  in  raves  is  likely  to  have  been  underreported  by 
study  participants  due  to  a  fear  of  reprisal  by  the  chain  of  command  at  NTC  Great  Lakes. 
While  participants  were  assured  anonymity  at  the  beginning  of  the  study,  many  may 
have  had  continuing  fears  that  leadership  might  take  action  against  them  or  their 
shipmates  based  on  what  they  said. 

When  questioned  about  transportation  on  and  off  base,  the  majority  of  participants 
indicated  that  their  main  concern  was  obtaining  cost-effective  and  safe  transportation 
when  traveling  off  base.  The  most  common  off-base  transportation  used  by  Sailors  was 
the  “Jam  Vans.”  These  are  independent  companies  providing  transportation  for  Navy 
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personnel  and  others  in  the  community,  similar  to  a  taxi  service,  utilizing  vans  instead 
of  cars.  This  mode  of  transportation  was  cited  as  being  expensive,  unreliable,  and 
potentially  dangerous.  Sailors  reported  incidents  of  Jam  Vans  not  always  coming  back 
to  pick  them  up  after  dropping  them  off.  The  focus  group  participants  also  reported 
other  incidents  of  poor  driving  behavior  and  the  presence  of  drugs/alcohol.  They  also 
indicated  that  on-base  transportation  was  poor  and  unreliable,  especially  for  junior 
enlisted  Sailors. 

Overall,  Sailors  overwhelmingly  preferred  to  be  off  base  whenever  possible  for  their 
off-duty  activities.  The  most  common  off-duty  activities  cited  by  junior  enlisted  Sailors 
were  going  to  the  local  shopping  malls,  visiting  the  city  of  Chicago,  or  visiting 
friends/family  in  the  area  around  the  base. 
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Summary  and  Conclusions 
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•  Present  study  sought  to  compare  traditional  face-to-face  with  two  on¬ 
line  focus  group  methods  to  evaluate  the  differing  levels  of  social 
presence  and  the  overall  success  of  the  approach. 

•  While  differences  were  found  in  the  number  of  words  used  to 
communicate,  the  three  conditions  appeared  to  achieve  similar  results 
in  level  of  participation,  participant  satisfaction,  and  quantity/quality  of 
information  obtained. 

-  The  only  exception  to  this  finding  was  that  participants  in  the  on¬ 
line  conditions,  especially  the  computer  mediated  condition,  were 
more  likely  to  make  off-topic  comments  or  engage  in  conflict  with 
other  group  members  (e.g.,  insults,  confrontation,  etc.). 

•  In  contrast  to  what  was  expected,  social  presence  did  not  appear  to 
clearly  distinguish  between  each  of  the  different  modes  of 
participation. 

Even  though  the  three  participation  modes  were  designed  to  be 
different  in  a  number  of  features,  participant  social  presence  ratings 
did  not  differ  as  predicted  by  theory. 

•  Overall  the  results  indicate  that  conducting  on-line  focus  groups  can  be 
done  in  a  successful  manner  and  can  achieve  similar  results  to  what  is 
typically  found  in  traditional  face-to-face  methods. 
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The  study  sought  to  compare  traditional  face-to-face  with  two  computer-based  focus 
group  methods  to  evaluate  the  differing  levels  of  social  presence  and  the  overall  success 
of  the  approach.  The  study  successfully  completed  27  focus  groups,  using  a  sample  of 
156  junior  enlisted  and  petty  officers  currently  in  training  at  the  NTC  Great  Lakes,  to 
evaluate  differences  in  the  success  of  FTF  and  CM  focus  groups.  Participants  in 
randomly  assigned  groups  of  5-6  participated  in  one  of  the  three  following  conditions: 
traditional  face-to-face  (standard  focus  group  utilizing  verbal  and  non-verbal 
communication);  computer-mediated  (group  members  in  same  location  but  all 
communication  via  computer);  and  Internet-simulated  (group  members  in  same 
location  but  separated  by  barriers  and  noise  canceling  headphones,  with  all 
communication  via  computer). 

While  differences  were  found  in  the  number  of  words  used  to  communicate,  the 
three  conditions  achieved  similar  results  in  level  of  participation,  participant 
satisfaction,  and  quantity/ quality  of  information  obtained.  The  only  exception  to  this 
finding  was  that  participants  in  the  on-line  conditions,  especially  the  computer- 
mediated  condition,  were  more  likely  to  make  off-topic  comments  or  engage  in  conflict 
with  other  group  members  (e.g.,  insults,  confrontation,  etc.).  This  finding  may  be  due  in 
part  to  the  hybrid  nature  of  the  computer-mediated  condition.  This  condition  was 
designed  to  be  a  combination  of  features  from  the  two  extremes  of  either  the  FTF 
condition  (e.g.,  high  social  presence  due  to  a  high  level  of  access  to  visual,  spatial,  and 
auditory  social  cues)  and  the  Internet-simulated  condition  (e.g.,  low  social  presence  due 
to  lack  of  access  to  visual,  spatial,  and  auditory  social  cues  because  participants  had  to 
rely  on  text  messages  only).  Even  though  participants  were  instructed  to  communicate 
only  via  the  computer,  by  not  blocking  off  access  to  visual,  spatial,  and  auditory  cues  in 
this  condition,  participants  had  access  to  much  more  information  than  in  other 
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conditions.  A  possible  explanation  could  be  the  fact  that  participants  were  able  to  see 
the  reactions  of  others  if  they  cracked  a  joke  or  insult  via  the  computer.  The  facial 
expressions  and  laughter  as  a  result  of  such  comments  encouraged  and  reinforced  this 
behavior  further  adding  to  the  amount  of  disagreement  and  conflict. 

In  contrast  to  what  was  expected,  social  presence  did  not  distinguish  between  the 
different  modes  of  participation.  Even  though  the  three  participation  modes  were 
designed  to  be  different  on  a  number  of  features,  participant  social  presence  ratings  did 
not  differ.  It  is  unclear  why  the  study  failed  to  find  differences  in  the  level  of  social 
presence.  There  may  be  a  need  for  further  development  of  the  concept  of  social 
presence. 

Overall  the  results  indicate  that  conducting  on-line  focus  groups  can  be  done  in  a 
successful  manner  and  achieve  similar  results  as  typically  found  in  traditional  face-to- 
face  methods.  While  there  are  some  differences  in  the  amount  of  conflict  and  words 
used  to  communicate,  similar  amounts  of  unique  information  are  obtained  for  both  FTF 
and  CM  focus  groups. 
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Advantages 

Experimental  Design.  Allowed  researchers  to  draw  conclusions  about  the 
impact  of  various  types  of  focus  groups  on  differences  in  quantity  and 
quality  of  information  obtained  and  group  process  factors. 

Larger  Sample  Size.  The  use  of  a  significantly  larger  sample  size  than  in 
previous  studies  allowed  for  greater  statistical  power  in  the  analyses. 
Synchronous  Chat-room  Approach.  Tested  a  synchronous  chat-room 
method  of  conducting  focus  groups  which  provided  a  near-realistic  test  of 
what  could  be  used  by  employers  in  the  current  work  environment. 

•  Limitations 

No  Follow-up.  The  study  will  not  conduct  any  follow-up  on  the  validity  of 
the  content  collected  during  the  focus  groups— we  really  have  no  way  of 
knowing  whether  or  not  participants  were  honest  during  the  group 
discussions. 

Simulated  vs.  "Real  life  ”  Laboratory  simulation  may  be  different  enough 
from  "real  life"  so  that  the  results  may  not  represent  what  could  be 
expected  if  tested  in  the  Fleet.  Simulation  used  an  Intranet  which  did  not 
offer  "real  life"  features  such  as  the  ability  to  use  the  Internet  browser  or 
other  software  to  get  out  of  the  experimental  situation  and  engage  in  other 
tasks  (e.g.,  browse  the  Internet,  receive  e-mail,  etc.). 


Novelty  of  the  Experiment.  The  novelty  of  the  experiment  may  have 
negatively  impacted  measurement  of  participant  satisfaction.  Notes  from 
the  moderators  indicate  that  participants  often  have  restricted  use  of 
computers  at  their  base. 


The  findings  indicate  that  overall  results  from  the  two  different  types  of  focus  groups 
were  similar  in  the  quantity  and  quality  of  information  produced.  In  contrast  to  what 
was  expected,  perceived  social  presence  did  not  differ  between  the  conditions. 

The  present  study  has  both  strengths  and  weaknesses.  Some  of  the  strengths  of  this 
study  included  an  experimental  design,  a  large  sample  size,  and  a  synchronous  chat 
room.  Unlike  many  previous  studies,  the  present  study  utilized  an  experimental  design. 
Study  conditions  were  carefully  constructed  and  administered  to  a  sample  of  Sailors 
who  were  randomly  assigned  by  group  to  participate  in  one  of  three  different  modes. 
This  experimental  design  made  the  present  study  significantly  more  useful  in  evaluating 
differences  in  FTF  and  CM  focus  groups  by  controlling  a  wide  variety  of  sources  of  error 
(e.g.,  selection  bias,  time,  order  effects).  Second,  this  study  used  a  significantly  larger 
sample  size  than  has  been  typically  used  in  studies  of  this  topic.  Many  previous  studies 
comparing  FTF  and  CM  have  used  only  2-4  groups,  with  4-8  members  in  each  group. 
This  study  utilized  9  groups  in  each  of  the  3  conditions  with  5-6  members  in  each 
group.  The  larger  sample  size  used  in  this  study  ensures  that  results  can  be  generalized 
across  the  experiment  and  that  statistical  tests  would  achieve  adequate  levels  of  power 
(Cohen,  1992)  to  detect  differences  in  average  responses  where  they  exist.  Third,  the 
study  tested  a  new  synchronous  chat  room  method  of  conducting  focus  groups.  At 
present,  with  a  little  over  90  percent  of  officers  and  80  percent  of  enlisted  Sailors 
reporting  access  to  the  Internet  (Olmsted  &  Underhill,  2003),  it  is  important  to  utilize  a 
method  of  conducting  on-line  communication  that  is  viable  in  today’s  Navy.  The 
promises  of  real-time  audio  and  video  communications  over  the  Internet  to  any  Sailor 
or  Marine  through  the  Navy/Marine  Corps  Intranet  are  in  the  future.  Today,  utilizing  a 
text-based  method  of  communication  (e.g.,  synchronous  chat  room)  researchers  can 
realistically  conduct  focus  groups  over  the  Internet,  and  should  be  able  to  reach  out  to 
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the  over  80  percent  of  the  Fleet  who  report  access  to  the  Internet  today.  In  the  future,  as 
the  on-line  capabilities  of  the  Fleet  are  improved,  other  methods  of  conducting  focus 
groups  over  the  Internet  can  be  tested  and  implemented. 

Some  of  the  disadvantages  of  the  study  included  the  lack  of  a  follow-up,  the 
simulated  vs.  real-life  nature  of  this  experiment,  and  the  novelty  of  the  experiment  for 
the  Sailors  who  participated  in  the  study.  First,  one  of  the  major  drawbacks  to 
conducting  focus  groups  as  a  method  of  data  collection  is  that  they  do  not  provide  a 
mechanism  to  do  follow-up  on  the  content  or  thematic  data.  The  moderators  who  lead 
each  group  do  ask  participants  to  be  truthful  and  forthcoming  in  their  responses  so  that 
they  encourage  honesty  in  the  process  of  conducting  focus  groups.  However,  moderators 
and  others  typically  do  not  go  back  to  participants,  their  work  supervisors,  or  others  and 
check  to  see  if  the  responses  are  valid  and  reliable  in  relation  to  what  is  being  reported. 
Research  conducted  by  Clapper  and  Massey  (1996)  suggests  that  on-line  communication 
provides  an  environment  where  people  are  free  to  express  their  opinions  and  that  they 
are  generally  honest  and  forthcoming;  however,  the  truthfulness  of  responses  remains 
to  be  clearly  demonstrated.  Second,  the  study  utilized  an  artificial  laboratory  setting  to 
conduct  the  focus  groups.  The  researchers  specifically  tried  to  find  a  setting  for  the  focus 
groups  that  might  simulate  as  closely  as  possible  the  conditions  under  which  focus 
groups  would  take  place  out  in  the  Fleet.  The  exception  to  this  was  that  the  Internet- 
simulated  condition  had  to  rely  on  simulating  the  separation  that  would  be  caused  by 
participating  on  the  Internet,  rather  than  actually  having  the  Sailors  participate  at 
separate  locations  throughout  the  base.  In  addition,  participants  were  blocked  from 
accessing  many  of  the  real-life  features  of  their  computer-workstations  or  the  Internet 
by  controls  placed  on  the  computers  by  the  research  team.  The  net  result  of  the  artificial 
nature  of  the  laboratory  setting  may  have  adversely  impacted  the  results  of  this  study. 
Future  studies  will  need  to  evaluate  the  impact  of  other  and  possibly  more  realistic 
settings  in  which  to  conduct  focus  groups.  Third,  the  novelty  of  being  pulled  from 
classes  and  being  asked  to  participate  in  a  research  study  may  have  in  some  way 
changed  the  behavior  of  participants. . 
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Future  Directions 
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•  "Real  Life”  Testing 

Do  further  testing  of  the  chat-room  approach  to  conducting  focus 
groups  utilizing  more  "real  life”  setting  such  as  the  Sailor’s  own 
computer  at  their  worksite.  This  will  allow  for  a  more  realistic  test 
of  the  process  of  conducting  remote  Internet-based  focus  groups. 

•  Test  Other  Populations 

J  unior  enlisted  Sailors  may  be  systematically  different  than  other 
demographic  groups  within  the  lMavy.  Research  indicates  that 
younger  members  of  the  U.S.  population  tend  to  have  more 
experience  with  computers  and  chat-rooms. 

•  Expand  the  Social  Presence  Literature 

-  The  design  of  conditions  and  measures  of  Social  Presence  used  for 
this  study  were  developed  using  the  existing  published  literature. 
Since  there  is  a  significant  lack  of  literature  on  the  subject,  the 
measures  and  conditions  may  not  accurately  represent  the 
construct  as  originally  intended  by  the  originators  of  social  presence 
theory.  Further  work  should  be  done  on  defining  the  construct  in  a 
wider  variety  of  settings  and  delineating  the  impact  that  it  has  on 
the  communication  process. 
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A  review  of  the  findings  of  this  study  points  towards  a  number  of  things  that  will  be 
necessary  to  validate  the  use  of  the  chat  room  format  or  other  methods  of  on-line  focus 
groups.  First,  there  is  a  distinct  need  for  more  real-life  testing.  Experimental  testing  in 
the  laboratory  setting  continues  to  be  needed  to  explore  the  limits  of  this  format  of 
conducting  focus  groups.  However,  the  artificial  nature  of  this  approach  makes  it  poorly 
suited  to  adequately  evaluate  the  method  in  the  real-life  settings  that  most  people  are 
likely  to  be  in  when  called  upon  to  participate  in  on-line  focus  groups.  For  instance, 
when  research  issues  arise  about  operational  problems  in  the  Navy,  the  Sailors  that 
researchers  may  want  to  reach  most  will  not  always  be  able  to  come  to  a  lab  to  take  part 
in  a  study.  Instead,  the  chat  room  format  is  designed  to  allow  Sailors  or  others  to 
participate  at  their  worksite,  in  computer  labs,  or  at  home  regardless  of  their  location. 
However,  since  testing  has  only  been  done  so  far  in  a  laboratory  setting,  there  is  no 
information  available  about  the  practical  limits  of  the  approach.  Issues  such  as  selection 
of  participants,  real-time  access  to  the  Internet,  electronic  security,  network  bandwidth 
constraints,  validating  that  users  are  who  they  say  they  are,  and  others  will  all  need  to  be 
worked  out  in  the  future.  We  believe  that  this  method  shows  good  promise  for  future 
applications;  however,  significant  real-life  testing  will  be  needed  to  demonstrate  this. 

Second,  testing  of  this  methodology  needs  to  be  extended  to  a  wider  variety  of 
populations  to  ensure  that  it  can  be  broadly  implemented  as  a  research  method.  The 
junior  enlisted  Sailors  who  made  up  the  sample  of  the  present  study  were  chosen 
primarily  because  they  were  an  easy  group  for  researchers  to  gain  access  to.  It  will  be 
important  to  demonstrate  that  the  method  works  well  on  a  wide  variety  of  other 
demographic  groups.  For  instance,  one  of  the  reasons  why  the  computer-based  focus 
groups  in  this  study  worked  so  well  may  have  been  because  most  of  the  participants  had 
previous  computer,  Internet,  and  chat  room  experience.  Other  groups  of  people  either  in 
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the  Navy  or  general  population  that  have  less  exposure  to  or  knowledge  of  computers, 
the  Internet,  and  chat  rooms  may  have  more  difficulty  in  responding  in  focus  groups 
that  are  computer-based. 

Third,  there  is  a  distinct  need  for  the  further  development  and  expansion  of  the 
theory  of  social  presence  in  the  scientific  literature.  At  the  present,  very  little  research  on 
the  concept  of  social  presence  has  been  published.  A  few  studies  have  mentioned  the 
concept  that  was  originally  proposed  by  Short  et  al.  (1976);  however,  none  have 
appreciably  extended  the  theory.  At  the  present,  there  is  a  need  to  further  develop  the 
theory  and  measures  of  the  construct,  or  to  find  a  suitable  replacement  to  help 
researchers  explain  differences  in  the  social  context  of  different  types  of  communication. 
The  measures  and  design  of  conditions  for  this  study  were  developed  based  on  the  small 
amount  of  existing  literature.  Since  there  is  a  significant  lack  of  literature  on  the  subject, 
the  measures  and  conditions  may  not  accurately  represent  the  construct  as  intended  by 
the  originators  of  social  presence  theory.  Further  work  should  be  done  on  defining  the 
construct  in  a  wider  variety  of  settings  and  delineating  the  impact  that  it  has  on  the 
communication  process. 
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INSTRUCTIONS  FOR  RECRUITING  PARTICIPANTS 
FOR  THE  FOCUS  GROUP  STUDY 


BACKGROUND:  This  study  seeks  to  expand  on  previous  research  comparing  face-to-face  and  computer-based 
methods  of  conducting  focus  groups.  Focus  groups  are  structured  group  discussions  designed  to  collect  information 
from  participants  on  their  attitudes  and  opinions  on  a  specific  subject.  As  use  of  computers  and  the  Internet  becomes 
more  widespread,  traditional  research  activities  such  as  focus  groups  are  being  shifted  to  new  computer-based 
approaches.  Recent  research  has  produced  mixed  results  with  some  studies  finding  the  methods  equivalent  and 
others  not  equivalent.  This  study  will  evaluate  the  comparability  of  traditional  face-to-face,  computer-mediated  (group 
members  in  same  location  but  all  communication  via  computer),  and  Internet-based  (group  members  in  different 
locations  and  all  communication  via  computer)  focus  groups.  The  results  of  the  study  will  help  Navy  researchers 
determine  if  they  are  likely  to  receive  useful  and  equivalent  responses  from  Sailors  when  conducting  computer- 
mediated  or  Internet-based  focus  groups. 

SPONSOR:  This  study  is  jointly  sponsored  by  the  the  Independent  Laboratory  Independent  Research  (ILIR)  program, 
from  the  Office  of  Naval  Research  and  the  Training  Analysis  Cell  there  at  the  Naval  Training  Center  Great  Lakes. 

SAMPLE:  This  study  will  use  a  sample  of  junior  enlisted  Sailors  (E-1  to  E-3)  and  petty  officers  (E-4  to  E-6)  who  are 
currently  in  training  programs  at  the  Naval  T raining  Center  Great  Lakes.  A  total  of  9  focus  groups  (6  members  each)  in 
each  of  three  conditions  will  be  conducted  resulting  in  a  total  sample  of  162  Sailors. 

POINT  OF  CONTACT:  Murrey  Olmsted,  Navy  Personnel  Research,  Studies,  &  Technology  Department, 
(901)  874-2130  commercial  telephone,  (DSN)  882-2130,  or  murrey.olmsted@persnet.navy.mil  (e-mail). 


INSTRUCTIONS:  Please  read  the  material  below  to  any  of  the  Sailors  within  your  program  that  may  be 
able  or  interested  in  participating  in  this  study.  We  are  seeking  volunteers  for  this  study,  so  please  do  not 
in  any  way  force  Sailors  to  be  a  part  of  the  study.  After  reading  the  material  below,  have  any  interested 
Sailors  sign-up  for  a  time-slot  on  the  attached  sheet. 


To  Be  Read  To  Potential  Study  Participants: 

Over  the  next  two  weeks,  researchers  from  the  Navy  Personnel  Research  Studies  and  Technology 
Department  at  the  Navy  Personnel  Command  will  be  conducting  a  number  of  focus  groups  here  at  the 
Naval  Training  Center  Great  Lakes.  These  focus  groups  will  focus  on  two  objectives.  First,  to  collect 
information  on  what  types  of  on-base  and  off-base  recreation  activities  Sailors  like  to  participate  in  when 
they  are  off-duty.  This  information  is  being  collected  to  assist  policy  makers  in  the  process  of  reviewing 
current  programs  and  evaluating  potential  changes  to  programs  and  services.  Second,  the  researchers 
will  be  testing  the  comparability  of  face-to-face  and  computer-based  methods  of  conducting  focus  groups. 
This  is  being  done  to  facilitate  the  on-going  integration  of  computers  into  the  every-day  work  lives  of 
Sailors. 

The  study  will  require  the  participation  of  a  total  of  162  enlisted  Sailors.  The  researchers  will  be  here 
conducting  focus  groups  from  12-21  March.  If  you  choose  to  participate,  you  will  be  asked  to  take  part  in 
a  focus  group  discussion  and  complete  a  short  questionnaire.  This  should  involve  approximately  1 .5 
hours  of  your  time.  All  individual  responses  will  be  kept  confidential.  The  group  discussions  are  being 
taped  recorded  (or  electronically  saved)  for  the  purpose  of  later  transcribing  and  compiling  the  data.  No 
identifying  information  will  be  used  so  comments  will  not  be  able  to  be  tracked  back  to  you  or  any  other 
participant.  In  addition,  this  study  is  completely  voluntary... so  you  cannot  be  forced  to  participate. 

If  you  would  like  to  participate  in  this  study,  please  see _ 

(insert  name  of  LPO  in  charge  here)  to  sign-up  for  a  time  slot. 
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Impact  of  Social  Presence  on  Focus  Group  Success 

Schedule  for  Data  Collection 

Naval  Training  Center  Great  Lakes 


22  April  2002  Monday 


Arrive  at  NTC  Great  Lakes 
Set  up  equipment 


Location:  Combat  Systems  A-Schools  Location:  Engineering  Systems  A-Schools 


23  April  2002  Tuesday 

Group 

Number 

Particpants 

Needed 

29  April  2002  Monday 

Group 

Number 

Particpants 

Needed 

8:00  to  10:00 

Group  1 

6 

8:00  to  10:00 

Set  up  equipment 

10:00  to  12:00 

Group  2 

6 

10:00  to  12:00 

Group  17 

6 

13:00  to  15:00 

Group  3 

6 

13:00  to  15:00 

Group  18 

6 

15:00  to  17:00 

Group  4 

6 

15:00  to  17:00 

Group  19 

6 

Group 

Particpants 

Group 

Particpants 

24  April  2002  Wednesday 

Number 

Needed 

30  April  2002  Tuesday 

Number 

Needed 

8:00  to  10:00 

Group  5 

6 

8:00  to  10:00 

Group  20 

6 

10:00  to  12:00 

Group  6 

6 

10:00  to  12:00 

Group  21 

6 

13:00  to  15:00 

Group  7 

6 

13:00  to  15:00 

Group  22 

6 

15:00  to  17:00 

Group  8 

6 

15:00  to  17:00 

Group  23 

6 

Group 

Particpants 

Group 

Particpants 

25  April  2002  Thursday 

Number 

Needed 

1  May  2002  Wednesday 

Number 

Needed 

8:00  to  10:00 

Group  9 

6 

8:00  to  10:00 

Group  24 

6 

10:00  to  12:00 

Group  10 

6 

10:00  to  12:00 

Group  25 

6 

13:00  to  15:00 

Group  11 

6 

13:00  to  15:00 

Group  26 

6 

15:00  to  17:00 

Group  12 

6 

15:00  to  17:00 

Group  27 

6 

17:00  to  18:00 

Pack  equipment 

Group 

Particpants 

Group 

Particpants 

26  April  2002  Friday 

Number 

Needed 

2  May  2002  Thursday 

Number 

Needed 

8:00  to  10:00 

Group  13 

6 

Return  to  NPRST  (Millington,  TN) 

10:00  to  12:00 

Group  14 

6 

13:00  to  15:00 

Group  15 

6 

15:00  to  17:00 

Group  16 

6 

17:00  to  18:00  Pack  Equipment 


POINT  OF  CONTACT:  If  you  have  any  questions  regarding  the  study  or  the  schedule,  please  contact  Murrey 
Olmsted,  (901 )  874-2130  /  (DSN)  882-2130  or  Christina  Underhill,  (901 )  874-2131 . _ 
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IMPACT  OF  SOCIAL  PRESENCE  ON  FOCUS  GROUP  SUCCESS 


Outline  of  Focus  Group  Protocol 

(Estimated  Total  Time:  52-60  minutes) 


Moderator  statement  and  introduction  ( 1-2  minutes) 

Thank  you  for  volunteering  in  today’s  focus  group  about  on-base  and  off-base  activities  here  at  Naval 
Training  Center  Great  Lakes.  The  focus  groups  that  we  will  be  conducting  in  the  next  two  weeks  are  part 
of  a  larger  study  that  will  compare  the  results  from  three  different  methods  of  focus  group  discussions.  I 
will  be  asking  you  all  a  series  of  questions  regarding  on-base  and  off-base  activities.  Murrey  Olmsted  or 
Christina  Underhill  will  be  the  assistant  moderator  who  will  be  taking  notes  on  the  discussion.  We 
encourage  you  to  discuss  among  each  other  the  answers  to  our  questions.  There  are  no  right  or  wrong 
answers,  but  we  do  expect  that  you  will  respect  the  opinions  of  others.  All  individual  responses  will  be 
kept  confidential.  The  group  discussions  are  being  taped  recorded  (or  saved)  for  the  purpose  of  later 
transcribing  and  compiling  the  data.  No  identifying  information  will  be  used  on  the  transcripts  so 
comments  will  not  be  able  to  be  tracked  back  to  you.  Your  participation  is  completely  voluntary  so  if  at 
any  time  you  feel  uncomfortable,  you  are  free  to  leave  without  any  explanation. 

Additional  Instructions  for  Computer  Mediated  Conditions 

Computer  Mediated  Group 

Please  type  your  first  name  in  the  log-in  prompt  on  the  laptop  in  front  of  you.  Once  you  have  typed  in 
your  name,  please  click  the  “log-in”  button  and  proceed  to  the  electronic  chat  room.  Please  respond 
to  the  questions  and  provide  any  input  on  the  discussion  by  typing  in  your  response  on  the  keyboard. 
When  you  have  finished  a  response,  please  press  the  “return”  key  to  post  your  response. 

Internet  Simulation  Group 

Use  the  same  as  additional  instructions  as  above,  with  the  addition  of  the  following: 

Now  put  on  and  adjust  your  headphones  to  make  sure  they  fit  securely  over  your  ears.  Please  keep 
the  headphones  on  your  ears  at  all  times  during  the  group  discussion.  Your  moderator  will  let  you 
know  when  you  can  take  the  headphones  off  and  when  discussion  is  complete. 


1.  Tell  us  your  first  name,  which  A-school  program  you  are  in,  and  what  you  do  in  your  spare 
time? 

(This  is  a  question  to  introduce  the  participants.  It  will  get  them  talking,  and  thinking  about  what  they 
do  in  their  “free”  time.) 


(4  minutes) 


2.  What  on-base  activities  do  you  use? 

•  What’s  appealing  about  these  activities? 

•  What’s  unappealing  about  these  activities? 

(This  is  a  non-threatening  question  that  will  be  easy  for  the  participants  to  answer.  Answers  to  this 
question  will  give  researchers  an  idea  of  what  the  Sailors  like  and  dislike  about  on-base  activities.) 


B-i 


(8-10  minutes) 


3.  What  are  some  of  your  favorite  places  to  go  off-base? 

•  How  often  do  you  go  off-base? 

•  What  are  some  hot  spots  people  go  to,  that  you  have  heard  of? 

(When  the  Sailors  go  off  base  they  have  a  tendency  to  get  into  more  trouble  (i.e.  fights,  public 
intoxication,  drug  use).  This  will  help  us  understand  what  off-base  activities  are  appealing  and  the 
frequency  they  are  attended.) 


(10  minutes) 

4.  We  have  heard  that  a  common  way  for  Sailors  to  get  off  base  is  to  use  “Jam  Vans”.  Tell  us 
your  experiences  with  the  Jam  Vans. 

(Jam  Vans  are  big  taxis  that  can  transport  groups  of  people.  They  are  allowed  on  base  so  that  the 
Navy  has  some  means  of  regulating  them.  There  have  been  complaints  about  the  service.  There  is 
also  an  idea  that  they  may  be  a  source  of  drugs  and  alcohol  on  base.  Several  sexual  assaults  have 
taken  place  on  the  Jam  Vans.) 

(10  minutes) 

5.  We  heard  that  raves  are  popular  in  the  greater  Chicago  area.  What  have  you  heard  about 
these  events?  What  kinds  of  things  have  you  heard  about  or  experienced  at  the  typical  rave? 

•  Have  you  or  any  of  your  friends  gone  to  a  rave? 

•  How  often  are  they  held? 

•  Have  you  ever  received  an  invitation  to  a  rave?  If  so,  how  did  receive  it. 

(Drinking  and  drug  use  are  common  at  raves.  Researchers  know  that  Sailors  go  to  them,  but  have  no 
idea  about  what  really  goes  on  or  how  the  Sailors  find  out  about  them. ) 

(15-20  minutes) 

6.  Is  there  anything  else  that  you  would  like  to  share  about  the  topic  that  you  didn’t  get  a  chance 
to? 


(4  minutes) 


Primary  Research  Questions  Requested  by  NTC  Great  Lakes 

•  Why  are  Sailors  opting  to  go  off  base  to  raves  and  other  places  and  engaging  in  risky  behavior? 

•  What  is  different  about  on-base  and  off-base  activities,  and  what  can  be  done  for  on-base  activities  to 
make  them  more  appealing? 

The  rationale  behind  this  line  of  questioning  would  be  if  you  could  make  the  on-base  activities  more 
appealing  than  they  wouldn’t  want  to  go  off-base  and  engage  in  risky-behavior. 


Back-up  Questions 

Besides  the  Prevent  program,  do  any  of  your  leaders  or  senior  personnel  give  you  advice  about  your 
lifestyle,  values  or  behavior  choices? 

•  Who  tends  to  give  this  advice  or  counseling? 

•  What  topics  or  subjects  does  this  advice  or  counseling  tend  to  be  about? 

•  When  does  this  advice  or  counseling  take  place? 

•  Where  does  this  advice  or  counseling  take  place? 
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ISP 

Participation  Consent  Form 
(and  Privacy  Act  Statement) 

Public  Law  93-579,  called  the  Privacy  Act  of  1 974,  requires  that  you  be  informed  of  the  purposes  of  this 
study  and  of  the  uses  to  be  made  of  the  information  collected.  The  authority  to  collect  this  information  is 
granted  under  Title  10,  United  States  Code,  sections  136  and  2358.  Please  read  the  following  details 
regarding  the  study  and  sign  at  the  bottom  of  this  page  if  you  wish  to  participate.  Please  ask  any 
questions  you  may  have  before  signing  this  form. 

1.  Background/Purpose.  You  are  invited  to  participate  in  the  present  study,  which  seeks  to  expand  on 
previous  research  comparing  face-to-face  and  computer-based  methods  of  conducting  focus  groups. 
The  results  of  the  study  will  help  Navy  researchers  determine  if  they  are  likely  to  receive  valid  and 
reliable  responses  from  Sailors  when  conducting  Internet-based  focus  groups 

2.  Information.  The  project  is  sponsored  by  the  Independent  Laboratory  Independent  Research  (ILIR) 
program  of  the  Office  of  Naval  Research.  This  program  allows  scientists  to  pursue  basic  and  applied 
research  on  topics  that  will  benefit  both  basic  science  and  Navy  researchers. 

3.  Procedures.  If  you  agree  to  participate,  you  be  asked  to  take  part  in  a  45-60  minute  focus  group 
discussion.  The  proceedings  of  this  focus  group  will  be  recorded  for  later  evaluation  by  the  research 
team  using  audio,  video,  or  electronic  taping  techniques.  Following  the  discussion,  you  will  be  asked 
to  complete  a  short  questionnaire  that  asks  about  your  background,  use  of  and  familiarity  with  the 
Internet,  and  a  few  questions  about  your  impressions  of  the  focus  group. 

4.  Routine  Uses:  The  information  you  provide  in  this  study  will  be  processed,  scanned  and  analyzed 
by  Navy  Personnel  Research,  Studies,  and  Technology  (NPRST).  The  data  provided  by  this  study 
will  be  used  to  determine  if  there  are  differences  in  the  impact  of  social  presence  on  the  quality  of 
data  collected  in  various  types  of  focus  groups. 

5.  Risks  and  Benefits.  This  research  involves  no  risks.  The  benefits  to  the  participation  are  that  you 
are  able  to  contribute  the  base  of  scientific  data  on  the  subject  of  computer-based  focus  groups. 

6.  Compensation.  No  tangible  reward  will  be  given. 

7.  Confidentiality.  All  of  your  responses  will  be  held  in  confidence  and  will  only  be  known  by  the 
researchers  involved  in  this  study  (i.e.  Murrey  Olmsted  and  Christina  Underhill).  This  includes  any 
information  collected  for  the  purposes  of  this  study  including  written,  audio,  video  or  electronic  data.  All 
data  records  will  use  random  4-digit  numbers  in  place  of  names  or  other  identifying  information  to 
protect  your  identity.  All  files  and  materials  associated  with  this  study  will  be  stored  in  a  secure  location 
and  any  electronic  files  will  stored  using  password  and  encryption  protection.  No  information  will  be 
publicly  accessible  which  could  identify  you  as  a  participant.  If  data  from  this  study  are  ever  released  to 
outside  sources,  all  identifying  information  will  be  withdrawn  from  all  files  to  protect  your  privacy. 

8.  Voluntary  Nature  of  the  Project.  If  you  agree  to  participate,  you  are  free  to  withdraw  from  the  study 
at  any  time  without  prejudice.  You  will  be  provided  a  copy  of  this  form  for  you  records  at  your 
request. 

9.  Points  of  Contact.  If  you  have  any  further  questions  or  comments  after  the  completion  of  this  study, 
you  may  contact  either  Murrey  Olmsted,  (901)  874-2130  or  Christina  Underhill,  (901)  874-2131 . 

10.  Statement  of  Consent.  I  have  read  the  above  information.  I  have  been  given  the  opportunity  to  ask 
any  questions  I  have  about  this  study  and  have  received  adequate  answers.  I  agree  to  participate  in 
this  study. 


Participant’s  Signature 


Date 


Investigator's  Signature 
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Date 


Group  ID _ 

Participant 

Name _ 

ISP 

Navy  Personnel  Research,  Studies,  and  Technology 
Millington,  TN 

The  present  study  seeks  to  expand  on  previous  research  comparing  face-to-face  and  computer- 
mediated  methods  of  conducting  focus  groups.  One  of  the  fundamental  strengths  of  focus  groups  is 
their  ability  to  create  a  sense  of  community  in  evaluating  and  discussing  a  topic.  This  occurs  in  large 
part  because  participants  work  together  as  a  group  in  the  same  location  where  they  can  see  and  hear 
each  other,  conveying  a  high  degree  of  social  presence.  As  use  of  computers  and  the  Internet  becomes 
more  widespread,  traditional  research  activities  such  as  focus  groups  are  being  shifted  to  new 
computer-mediated  approaches.  Recent  research  has  produced  mixed  results  with  some  studies 
indicating  equivalent  and  others  non-equivalent  social  presence  between  face-to-face  and  computer 
mediated  focus  groups.  In  addition,  while  many  researchers  point  to  the  promise  of  using  the  Internet 
for  focus  groups,  there  are  no  currently  available  experimental  studies  that  have  conducted  a  true 
Internet-based  focus  group  in  the  literature.  This  study  will  evaluate  the  impact  of  social  presence  on 
the  quantity  and  quality  of  participation  of  group  members  in  face-to-face,  computer-mediated  (group 
members  in  same  location  but  all  communication  via  computer),  and  Internet-based  (group  members  in 
different  locations  and  all  communication  via  computer)  focus  groups.  The  results  of  the  study  will 
help  Navy  researchers  detennine  if  they  are  likely  to  receive  valid  and  reliable  responses  from  Sailors 
when  conducting  computer-mediated  or  Internet-based  focus  groups 

Please  read  the  following  Privacy  Act  Statement  below  and  then  proceed  to  the  next  page  when  ready. 
The  survey  should  take  approximately  15  minutes  of  your  time  to  complete. 


Privacy  Act  Statement: 

Purpose:  Data  from  this  study  will  be  used  to  expand  on  previous  research  comparing  face-to-face  and  computer-based 
methods  of  conducting  focus  groups.  The  results  of  the  study  will  help  Navy  researchers  determine  if  they  are  likely  to 
receive  valid  and  reliable  responses  from  Sailors  when  conducting  Internet-based  focus  groups.  The  information  provided 
in  this  survey  will  be  analyzed  and  maintained  by  the  Navy  Personnel  Research,  Studies,  and  Technology  Department 
(PERS-1)  of  the  Navy  Personnel  Command. 

Routine  Uses:  None.  This  survey  data  will  be  used  on  a  one-time  basis  for  this  study  and  will  not  be  reused  in  any  form. 

Confidentiality:  All  responses  will  be  secured  and  held  in  confidence  by  the  Navy  Personnel  Research,  Studies,  and 
Technology  Department  (PERS-1).  Information  you  provide  will  only  be  considered  when  statistically  combined  with  the 
responses  of  others,  and  will  NOT  be  identified  with  any  single  individual.  This  survey  will  also  NOT  use  cookies  or  any 
other  electronic  tracking  devices  to  maintain  records  of  individual  respondents. 

Participation:  Completion  of  this  survey  is  entirely  voluntary.  Failure  to  respond  to  any  of  the  questions  will  NOT  result 
in  any  penalties  except  for  a  lack  of  representation  of  your  views. 
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Your  Background 


1 .  What  is  your  current  age? 


2.  What  is  your  gender? 

□  Male 

□  Female 

3.  What  is  your  paygrade? 

□  E-l  □  E-2  □  E-3 

□  E-4  □  E-5  □  E-6 

□  E-7  □  E-8  □  E-9 


How  much  to  you  AGREE  or  DISAGREE  with  the  following 
statements  regarding  your  experience  and  comfort  with  using 
computers. 

Strongly 

Agree 

Agree 

Neither 

Disagree 

Strongly 

Disagree 

5.  I  have  experience  using  computers 

□ 

□ 

□ 

□ 

□ 

6.  I  am  comfortable  using  computers 

□ 

□ 

□ 

□ 

□ 

7.  I  have  experience  using  Internet  chat  or  instant  messaging  software 
(i.e.  AOL  instant  messenger,  MSN  messenger,  etc.) 

□ 

□ 

□ 

□ 

□ 

8.  I  am  comfortable  using  Internet  chat  or  instant  messaging  software 
(i.e.  AOL  instant  messenger,  MSN  messenger,  etc.) 

□ 

□ 

□ 

□ 

□ 

9.  How  long  have  you  been  using  the  Internet  (for  browsing,  e-mail  or  other  use)? 

□  Does  not  apply;  I  have  not  used  the  Internet 

□  Less  than  1  year 

□  1  year  to  less  than  2  years 

□  2  years  to  less  than  3  years 

□  3  years  to  less  than  4  years 

□  4  or  more  years 

10.  In  an  average  week,  how  often  do  you  use  the  Internet  (for  browsing,  e-mail  or  other  use)? 

□  Does  not  apply;  I  do  not  have  Internet  access 

□  Never 

□  Once  a  week 

□  Several  times  a  week 

□  Once  a  day 

□  Several  times  a  day 
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Focus  Group  Evaluation 

Instructions:  Please  answer  the  following  questions  regarding  your  participation  in  the  focus  group. 


How  much  do  you  AGREE  or  DISAGREE  with  the  following 
statements  regarding  how  well  topics  were  covered  in  the  group 
discussion. 

1 .  The  group  discussed  the  topic  thoroughly 

Strongly 

Agree 

Agree 

Neither 

Disagree 

Strongly 

Disagree 

□ 

□ 

□ 

□ 

□ 

2. 

I  did  not  have  an  opinion  about  most  of  the  topics  raised 

□ 

□ 

□ 

□ 

□ 

3. 

The  group  had  a  lot  of  good  ideas  about  the  topic 

□ 

□ 

□ 

□ 

□ 

4. 

The  discussion  frequently  got  off  topic 

□ 

□ 

□ 

□ 

□ 

5. 

People  in  the  group  gave  their  honest  opinion 

□ 

□ 

□ 

□ 

□ 

6. 

The  subject  matter  of  the  group  was  interesting 

□ 

□ 

□ 

□ 

□ 

How  much  do  you  AGREE  or  DISAGREE  with  the  following 
statements  regarding  the  interaction  during  the  group  discussion. 

Strongly 

Agree 

Agree 

Neither 

Disagree 

Strongly 

Disagree 

7.  This  approach  was  appropriate  for  the  discussion  topic 

□ 

□ 

□ 

□ 

□ 

8.  I  had  a  chance  to  contribute  all  that  I  wanted 

□ 

□ 

□ 

□ 

□ 

9.  It  was  easy  to  talk  to  others  in  the  group 

□ 

□ 

□ 

□ 

□ 

10.  All  members  of  the  group  talked  about  the  same  amount 

□ 

□ 

□ 

□ 

□ 

1 1 .  It  was  easy  to  share  all  my  ideas 

□ 

□ 

□ 

□ 

□ 

12.  The  discussion  was  difficult  to  follow 

□ 

□ 

□ 

□ 

□ 

13. 1  am  comfortable  in  group  discussions 

□ 

□ 

□ 

□ 

□ 

14.  Other  participants  appeared  to  understand  my  comments 

□ 

□ 

□ 

□ 

□ 

15.1  felt  like  I  was  part  of  the  group 

□ 

□ 

□ 

□ 

□ 

16. 1  had  a  chance  to  explain  myself  when  I  felt  others  didn’t 
understand 

□ 

□ 

□ 

□ 

□ 

How  much  do  you  AGREE  or  DISAGREE  that  the  following 
attributes  were  demonstrated  in  the  group  discussion. 

17.  Personal  (as  opposed  to  impersonal) 

Strongly 

Agree 

Agree 

Neither 

Disagree 

Strongly 

Disagree 

□ 

□ 

□ 

□ 

□ 

18.  Sensitive  (as  opposed  to  insensitive) 

□ 

□ 

□ 

□ 

□ 

19.  Warm  (as  opposed  to  cold) 

□ 

□ 

□ 

□ 

□ 

20.  Sociable  (as  opposed  to  unsociable) 

□ 

□ 

□ 

□ 

□ 

21.  Formal  (as  opposed  to  informal) 

□ 

□ 

□ 

□ 

□ 

22.  Immediate  (as  opposed  to  delayed) 

□ 

□ 

□ 

□ 

□ 

23.  Comfortable  (as  opposed  to  uncomfortable) 

□ 

□ 

□ 

□ 

□ 

24.  Connected  (as  opposed  to  disconnected) 

□ 

□ 

□ 

□ 

□ 
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How  much  do  you  AGREE  or  DISAGREE  with  the  following 
statements  regarding  how  satisfied  you  were  with  the  group 
discussion. 

25.  There  were  times  when  I  was  not  sure  what  was  expected  of  me 

Strongly 

Agree 

Agree 

Neither 

Disagree 

Strongly 

Disagree 

□ 

□ 

□ 

□ 

□ 

26.  There  were  times  when  I  didn’t  know  what  to  say 

□ 

□ 

□ 

□ 

□ 

27. 1  would  be  willing  to  participate  in  another  group  like  this 

□ 

□ 

□ 

□ 

□ 

28. 1  felt  comfortable  giving  my  opinions 

□ 

□ 

□ 

□ 

□ 

29. 1  enjoyed  participating  in  this  focus  group 

□ 

□ 

□ 

□ 

□ 

30.  Overall  the  group  went  very  well 

□ 

□ 

□ 

□ 

□ 

Please  use  the  rating  scale  below  to  indicate  how  much  you  AGREE 
or  DISAGREE  with  how  each  statement  describes  you.  Describe 
yourself  as  you  generally  are  now,  not  as  you  wish  to  be  in  the  future. 
Please  read  each  statement  carefully,  and  then  mark  your  answer. 

31.1  am  the  life  of  the  party 

Strongly 

Agree 

Agree 

Neither 

Disagree 

Strongly 

Disagree 

□ 

□ 

□ 

□ 

□ 

32. 1  feel  comfortable  around  people 

□ 

□ 

□ 

□ 

□ 

33. 1  start  conversations 

□ 

□ 

□ 

□ 

□ 

34. 1  talk  to  a  lot  of  different  people  at  parties 

□ 

□ 

□ 

□ 

□ 

35. 1  don’t  mind  being  the  center  of  attention 

□ 

□ 

□ 

□ 

□ 

36. 1  don’t  talk  a  lot 

□ 

□ 

□ 

□ 

□ 

37. 1  keep  in  the  background 

□ 

□ 

□ 

□ 

□ 

38.1  have  little  to  say 

□ 

□ 

□ 

□ 

□ 

39. 1  don’t  like  to  draw  attention  to  myself 

□ 

□ 

□ 

□ 

□ 

40. 1  am  quiet  around  strangers 

□ 

□ 

□ 

□ 

□ 

41.1  make  friends  easily 

□ 

□ 

□ 

□ 

□ 

42. 1  take  charge 

□ 

□ 

□ 

□ 

□ 

43. 1  know  how  to  captivate  people 

□ 

□ 

□ 

□ 

□ 

44. 1  feel  at  ease  with  people 

□ 

□ 

□ 

□ 

□ 

45. 1  am  skilled  in  handling  social  situations 

□ 

□ 

□ 

□ 

□ 

46. 1  find  it  difficult  to  approach  others 

□ 

□ 

□ 

□ 

□ 

47. 1  often  feel  uncomfortable  around  others 

□ 

□ 

□ 

□ 

□ 

48. 1  bottle  up  my  feelings 

□ 

□ 

□ 

□ 

□ 

49. 1  am  a  very  private  person 

□ 

□ 

□ 

□ 

□ 

50. 1  wait  for  others  to  lead  the  way 

□ 

□ 

□ 

□ 

□ 

Thank  you  for  taking  your  time  to  complete  this  study. 
Your  input  is  greatly  valued. 
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ADDITIONAL  TRANSCRIPT  CODING  INSTRUCTIONS 


What  is  relevant  and  on  topic? 

Tell  us  your  first  name,  which  A-school  program  you  are  in,  and  what  you  do  in  your 
spare  time. 

On-base  activities 

What  on-base  activities  do  you  use?  What  is  appealing  or  unappealing  about  these? 
Off-base  activities 

What  are  some  of  your  favorite  places  to  go  off-base?  How  often  do  you  go  off  base? 
Transportation 

We  have  heard  that  a  common  way  for  Sailors  to  get  off  base  is  to  use  “Jam  Vans.’’  Tell 
us  your  experiences  with  the  Jam  Vans.  How  else  do  you  travel  off  base?  How  is  on  base 
transportation? 

Raves 

We  heard  that  raves  are  popular  in  the  greater  Chicago  area.  What  have  you  heard  about 
these  events?  What  kinds  of  things  have  you  heard  about  or  experienced  at  the  typical 
rave?  Have  you  or  any  of  your  friends  been  to  a  rave?  How  often  are  they  held?  Have  you 
ever  received  an  invitation  to  a  rave?  If  so,  how  did  you  receive  the  invitation? 

Quality  of  services  and  facilities 

Is  there  anything  else  that  you  would  like  to  share  about  on  base  or  off  base  activities?  If 
you  could  improve  one  things  about  on  base  activities  or  facilities  what  would  it  be? 

What  would  you  do  to  improve  life  here  on  base? 

The  Internet 

How  is  Internet  access  on  base? 

What  is  off  topic? 

Off-topic  comments  are  completely  unrelated  to  the  focus  group  topics  or  any  question 
posed  by  the  moderator.  Pleasantries,  small  talk,  comments  about  the  software,  the  room, 
or  the  moderators  themselves  are  also  off-topic. 

Coding  schemes 

(  )  :  On-topic  or  relevant  comments 
[  ]  :  Off-topic  comments 

Highlighted  yellow  :  Disagreements,  insults,  or  contradictions 
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Examples: 


A) 

86:  (an  attempt ) 

81:  (an  attempt) 

These  are  participation  attempts.  They  are  not  comments  and  are  neither  on  or  off  topic. 

B) 

181: _ (with  no  words) 

This  would  be  a  participation  attempt.  Not  a  comment. 

C) 

85: 1  go  on _ night.  OR  85:  1  go  on  ( )  night. 

Count  underlined  spaces  or  (  )  as  one  word.  This  is  where  I  was  unable  to  understand 
what  was  being  said. 

D) 

183:  From  600am  to  1430pm 

Count  am  and  pm  as  a  separate  word  so  the  above  sentence  is  6  words  long. 

E) 

213:  ok,  I  was  wrong  :-) 

Emoticans  (i.e.  are  smileys,  frowns,  etc)  are  counted  as  one  word. 

F) 

“Yeah  ”:  counts  like  a  yes  and  is  considered  a  comment 

“Oh  ”:  counts  as  a  word  or  comment  because  they  are  trying  to  communicate 

“ahhh  ”  or  “ummm  ”:  does  not  count  as  a  word  because  it  is  used  as  a  place  holder 

or  pause.  Not  really  communicating  unless  it  is  typed  in  an  online  focus  group.  If 
Ummm  is  typed  in  an  online  focus  group,  count  as  one  word  because  they 
deliberately  wanted  to  communicate  a  pause. 

“Ya  Know”:  counts  as  2  words 

“um  hum  ”:  counts  as  1  word  like  a  yes.  Statement  of  agreement. 

“huh  ”  and  “uh  ”  :  do  not  count  as  words 
“lol”  or  “/mao  ”  or  “rotfl”:  count  as  one  word 

G) 

( ):  I  don  7  know. 

This  is  where  I  couldn’t  tell  who  said  what.  Cross  out  and  don’t  count.  I  kept  it  in  as  a 
conversation  placeholder. 


Group  ID 
Rater  ID 


WHAT  DO  THEY  DO  IN  THEIR  SPARE  TIME?  GENERALLY  SPEAKING 

When  no  location  is  mentioned 


Hang  out;  relax 

Party/Kicking  it 

Drink 

Sleep 

Read 

Computer  games/playstation  2 

Workout/lift  weights 

Study 

Play  pool 

Eat 


ON  BASE  ACTIVITIES  AND/OR  FACILITIES 

Why  they  stay  on  base 

Stay  on  base  during  the  week  because  of  class,  study,  pt,  duty,  and  not  enough 
time  with  curfew  to  do  anything  off  base 
Save  money 


The  Pier  /  Club  525  /  Club  Liberty 
Alright  to  relax  and  hang  out  in 

Need  more  free  stuff  like  Internet,  video  games,  food,  and  pool 


Well  lit 


Ross  Theatre 

Free  movie  premiers 
Just  a  dollar 

Need  to  have  more  than  one  showing  of  the  premier 
Need  to  have  a  more  consistent  movie  schedule 
Need  movies  during  the  week 
Sound  systems  sucks 


The  track 


Group  ID 
Rater  ID 


Bowling  Alley 

Too  far  away,  long  walk 


The  Loft 

Have  free  pool 
Too  far  of  a  walk 
Not  well  lit 


uso 

Free  food,  games,  entertainment,  and  stuff 
Not  many  people  know  what  they  offer 
Not  well  lit 


Gyms 

Pretty  good 

Need  a  consistent  schedule  of  hours 
Don’t  stay  open  long  enough  in  evening 
Don’t  open  early  enough  in  the  morning 


Library 

Free  Internet 


Galley  /  Food  options 
Improve  the  food 
RTC  eats  better  than  they  do 
No  ability  to  cook  healthy  in  the  barracks 
Claim  they  don’t  eat  healthy  food  (those  that  don’t  eat  in  galley) 
Roommates  eat  your  food 
No  problem  with  the  galley 
Lines  are  too  long 

Doesn’t  stay  open  long  enough  in  the  evening 
More  food 


Group  ID _ 

Rater  ID  _ 

Run  out  of  advertised  food,  get  leftovers  or  not  a  full  or  balanced  meal 
Food  places  and  galley  are  adequate 
Would  like  more  variety  of  fast  food 
Jack  in  the  Box 
Burger  King 
Chick  Fila 
Popeye’s 
Arbys 
A  deli 


The  NEX  aka  Navy  Exchange 

Difference  in  price  for  on  and  off  base  Exchange 
Expensive  overall 

Intramural  games  or  sports 
Need  more 

Base  wide  teams  better  for  unity 
Sports  suggested: 

Male  and  female  basketball 

Soccer 

Tag  football 

Track  teams 

Boxing 

Baseball 


Need  a  skate  park  for: 
Rollerblading 
Skateboarding 


Cookouts/Barbecues 
Need  more 

Concerts/Live  bands 
Need  more 
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OFF  BASE  ACTIVITIES 


Group  ID 
Rater  ID 


Why  they  go  off  base 

To  get  away  from  Base 
To  get  away  from  BEQ 
To  get  away  from  other  students/need  space 
To  get  away  from  Navy 

Malls;  shopping 

Clubs 

Bars 

Parties  at  hotels 

Movies 

Restaurants 

Check  out  surrounding  area;  site  see 
Chicago 
Minneapolis 
Kenosha 
Milwaukee 
Waukegan 
Gurnie 

Chicago  is  an  expensive  city 

Pick  up  chicks/Look  for  women/girl  watch 


TRANSPORTATION 

On  base  shuttles 

Need  more  shuttles 

Don’t  see  them  often/or  when  you  need  them 
Usually  full 

They  don’t  stop  for  you 

Long  wait  for  them 

Need  more  shuttle  stops 

Need  a  schedule  of  stops 

Use  TPU  or  people  on  duty  for  drivers 

Use  a  school  bus  or  larger  vans 


Group  ID 
Rater  ID 


Jam  Vans 

You  get  a  good  driver  call  them  up 

Rates  change  at  their  discretion 

Rates  change  depending  on  the  number  of  people 

Get  stopped  by  the  cops  frequently 

Overall,  a  good  and  needed  service 

Driving  is  fast  and/or  dangerous 

Cheap  transportation 

Expensive  transportation 

Crowded  vans/too  many  passengers 

Should  be  allowed  on  base 

Some  vans  are  not  equipped  well  i.e.  seatbelts,  windshield  wipers 
Drivers  don’t  speak  good  English/hard  to  understand  them 
Best  way  to  travel  if  you  don’t  have  a  car  or  a  friend  with  a  car 
They  don’t  crash/haven’t  heard  of  them  crashing 
Some  good,  some  bad 


Personal  cars 

Need  more  parking  for  fleet  returnees 


Train 

Friend’s  car 


RAVES 

Been  to  a  rave  but  don’t  go  anymore 
Never  been  to  a  rave 
Go  regularly  to  raves 

All  about  music:  trance  and  techno,  hard  house 
Less  expensive  to  get  in  a  rave  than  a  club 
Not  about  the  drugs 
Lots  of  drugs  including  Exctasy 

What  is  a  rave?  Good  quotes  or  descriptions 
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Group  ID 
Rater  ID 


How  DO  THEY  HEAR  ABOUT  RAVES? 
Word  of  mouth 
Flyers,  advertisements 
Internet 
Underground 


INTERNET 

Library  has  free  intemet/30  minute  time  limit  at  the  library 

USO  has  free  internet/cheap 

Hard  to  get  access  at  USO 

Hard  to  get  access/can’t  use  it  that  long 

Long  wait  to  get  access  at  library 

Slow  loading  browsers  eat  up  Internet  time 

Limited  print  capabilities 

Expensive  to  check  email  in  room  with  current  phone  company 
Expensive  to  email/1  dollar  for  10  minutes 
Not  enough  computers 
Need  computers  in  BEQ 

Hard  to  communicate  with  family,  friends,  or  research  things  because  of  email  access  and 
cost 

Good  at  Pier 


CURFEW/LIBERTY 

Sucks/don’t  like  it/it’s  a  drag 
Need  more  free  time 
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B-SHIFT  ISSUES 


Group  ID 
Rater  ID 


Not  enough  free  time 


STAFF  ISSUES 

Poorly  motivated  staff/don’t  seem  to  care 
Yell  at  them  a  lot 


OTHER 

Phone  company  on  base  is  expensive 

Sailors  have  difficulty  saving  money  because  of  transportation  and  food  costs 
Sprint  retailer  doesn’t  provide  discount  to  military 
Gates  close  too  early  particularly  gate  4 
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Discussion  Content 


Unique  ideas  produced  in  response  to  core  questions.  Included  are  those  ideas  that 
were  mentioned  by  all  three  focus  group  types  and  at  least  8  times. 

What  do  you  do  in  your  spare  time? 

Number  of  focus  groups  contributing  unique  comments 


Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Watch  movies/dvds/TV 

7 

8 

7 

Hang  out/relax 

6 

6 

4 

Sleep 

5 

8 

4 

Workout/lift  weights 

8 

3 

4 

Read 

4 

5 

5 

Computer  games/playstation  2 

5 

4 

5 

Drink 

4 

4 

5 

Play  pool 

2 

6 

2 

Study 

4 

5 

2 

Hang  with  family 

5 

3 

0 

Sports 

4 

4 

2 

Total  Unique  Comments 

54 

56 

40 

Ross  Theatre 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

Know  about  it 

5 

7 

6 

Free  movie  premiers 

3 

3 

5 

Total  Unique  Comments 

8 

10 

11 
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The  Pier  /  Club  525  /  Club  Liberty 


Number  of  focus  groups  contributing  unique  comments 


Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Know  about  it 

7 

9 

7 

Alright  to  relax  and  hang  out  in 

4 

4 

2 

Total  Unique  Comments 

11 

13 

9 

The  Loft 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Know  about  it 

5 

4 

6 

Have  free  pool 

4 

4 

6 

Total  Unique  Comments 

9 

8 

12 

The  USO 


Number  of  focus  groups  contributing  unique  comments 


Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Know  about  it 

3 

4 

5 

Total  Unique  Comments 

3 

4 

5 
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The  Gyms 


Number  of  focus  groups  contributing  unique  comments 


Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Know  about  them 

8 

7 

9 

Pretty  good 

5 

6 

6 

Total  Unique  Comments 

13 

13 

15 

The  Library 


Number  of  focus  groups  contributing  unique  comments 


Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Know  about  it 

4 

1 

3 

Free  Internet 

3 

2 

3 

Never  open 

0 

1 

0 

Total  Unique  Comments 

7 

4 

6 

The  Galley  /  Food  Options 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Improve  the  food 

2 

6 

5 

Total  Unique  Comments 

2 

6 

5 

The  NEX 


Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Know  about  it 

4 

4 

3 

Total  Unique  Comments 

4 

4 

3 

Other  on  base  activities  /  facilities 


Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

The  Bowling  Alley  -  know  about  it 

5 

3 

5 

Total  Unique  Comments 

13 

13 

19 

Why  do  you  go  off  base? 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

To  go  home 

5 

2 

3 

To  get  away  from  Base 

2 

3 

3 

Total  Unique  Comments 

7 

5 

6 

What  are  some  of  your  favorite  places  to  go  off  base? 


Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

Chicago 

8 

8 

9 

Malls;  shopping 

7 

6 

8 

Clubs 

6 

5 

8 

Movies 

5 

5 

7 

Restaurants 

7 

4 

3 

Gumie 

5 

5 

8 

Hawthorne 

2 

0 

7 

Museums 

1 

5 

3 

Bars 

3 

3 

3 

Total  Unique  Comments 

44 

41 

56 

On  base  shuttles  /  transportation 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

Don’t  see  them  often/or  when  you  need  them 

3 

1 

5 

Want  an  off  base  shuttle 

2 

2 

4 

Long  wait  for  them 

3 

3 

3 

Total  Unique  Comments 

8 

6 

12 

Tell  us  your  experiences  with  the  Jam  Vans 


Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

based 

Computer- 

mediated 

Face-to- 

face 

Know  about  them 

8 

9 

8 

Expensive  transportation 

7 

8 

7 

Driving  is  fast  and/or  dangerous 

Rates  change  depending  on  the  number  of 

6 

7 

8 

people 

5 

4 

7 

Crowded  vans/too  many  passengers 

3 

7 

5 

Rates  change  at  their  discretion 

3 

3 

5 

Cheap  transportation 

4 

4 

1 

Overall,  a  good  and  needed  service 

3 

3 

2 

understand  them 

2 

4 

3 

Long  waits 

2 

4 

5 

Some  good,  some  bad 

3 

2 

5 

Total  Unique  Comments 

46 

55 

56 

Other  transportation 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

Take  the  train 

8 

7 

9 

Take  a  friend's  car 

4 

3 

6 

Take  a  taxi 

3 

3 

6 

Use  personal  cars 

4 

4 

3 

Walk 

4 

3 

1 

Total  Unique  Comments 

23 

20 

25 

E-6 


What  do  you  know  about  Raves? 


Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

Never  been  to  a  rave/stay  away  from  them 

8 

9 

5 

Lots  of  drugs  including  Exctasy 

8 

7 

5 

Been  to  a  rave  but  don’t  go  anymore 

6 

3 

5 

All  about  music:  trance  and  techno,  hard 

house 

3 

1 

4 

Total  Unique  Comments 

25 

20 

19 

How  do  you  hear  about  Raves? 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

Word  of  mouth 

3 

2 

6 

Total  Unique  Comments 

3 

2 

6 

Internet  Access 

Number  of  focus  groups  contributing  unique  comments 

Unique  comments 

Internet- 

Computer- 

Face-to- 

based 

mediated 

face 

Expensive  to  email/1  dollar  for  10  minutes 

4 

4 

5 

Library  has  free  interne t/30  minute  time  limit 

at  the  library 

3 

3 

4 

Hard  to  get  access/can’t  use  it  that  long 

5 

0 

2 

Total  Unique  Comments 

12 

7 

11 

Distribution 


AIR  UNIVERSITY  LIBRARY 

ARMY  MANAGEMENT  STAFF  COLLEGE  LIBRARY 

ARMY  RESEARCH  INSTITUTE  LIBRARY 

ARMY  WAR  COLLEGE  LIBRARY 

CENTER  FOR  NAVAL  ANALYSES  LIBRARY 

HUMAN  RESOURCES  DIRECTORATE  TECHNICAL  LIBRARY 

JOINT  FORCES  STAFF  COLLEGE  LIBRARY 

MARINE  CORPS  UNIVERSITY  LIBRARIES 

NATIONAL  DEFENSE  UNIVERSITY  LIBRARY 

NAVAL  HEALTH  RESEARCH  CENTER  WILKINS  BIOMEDICAL  LIBRARY 
NAVAL  POSTGRADUATE  SCHOOL  DUDLEY  KNOX  LIBRARY 
NAVAL  RESEARCH  LABORATORY  RUTH  HOOKER  RESEARCH  LIBRARY 
NAVAL  WAR  COLLEGE  LIBRARY 

NAVY  PERSONNEL  RESEARCH,  STUDIES,  AND  TECHNOLOGY  SPISHOCK 
LIBRARY  (3) 

PENTAGON  LIBRARY 

USAF  ACADEMY  LIBRARY 

US  COAST  GUARD  ACADEMY  LIBRARY 

US  MERCHANT  MARINE  ACADEMY  BLAND  LIBRARY 

US  MILITARY  ACADEMY  AT  WEST  POINT  LIBRARY 

US  NAVAL  ACADEMY  NIMITZ  LIBRARY 


